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1 Introduction 


This document is a report about the capabilities and performance of the IOA Toolkit, and in par- 
ticular the tools that provide support for implementing and running distributed systems (checker, 
composer, code generator). The Toolkit compiles distributed systems specified in IOA into Java 
classes, which run on a network of workstations and communicate using the Message Passing Inter- 
face (MPI). In order to test the toolkit, several distributed algorithms were implemented, ranging 
from simple algorithms such as LCR leader election in a ring network to more complex algorithms 
such as the GHS algorithm for computing the minimum spanning tree in an arbitrary graph. All 
of our experiments completed successfully, and several runtime measurements were made. 


2 Experiments 


Implementation Platform The machines used are located in the Theory of Computation Group 
of the MIT Computer Science and Artificial Intelligence Laboratory, forming a Local Area Network. 
They are all Red Hat Linux machines and with Intel Pentium III to IV with clock speed ranging 
from 1 GHz to 3.2 GHz. Even though MPI sets up a connection between every pair of nodes, the 
algorithms only use the communication channels they need. For example, a node (i) in LCR only 
sends to node i+1 and only receives from node i-1. 


2.1 LCR Leader Election 


The algorithm of Le Lann, Chang and Roberts for Leader Election in a ring network was the first 
experiment in running an IOA program on a network of computers. The automaton definition that 
appears in [1](Section 15.1) was used, with some modifications. For all the algorithms that follow, 
the nodes are automatically numbered from 0 to (size - 1). 


Automata Definitions The automata LCRProcess, LCRNode, SendMediator and ReceiveMe- 
diator were written. The mediator automata are given in Appendix C.1 (these automata implement 
the channel automata integrated with MPI functionality). The composed automaton (LCR) ap- 
pears in Appendix C.2 (this is the one that is translated into Java code by the IOA Toolkit). 


LCR Leader Election process automaton 


type Status = enumeration of idle, voting, elected, announced 


automaton LCRProcess(rank: Int, size: Int) 
signature 
input vote 
input RECEIVE(m: Int, const mod(rank - 1, size), const rank: Int) 
output SEND(m: Int, const rank: Int, const mod(rank+i, size)) 


output leader(const rank) 


states 
pending: Mset[Int] := {rank}, 
status: Status := idle 


transitions 
input vote 


eff status := voting 
input RECEIVE(m, j, i) where m > i 
eff pending := insert(m, pending) 


input RECEIVE(m, j, i) where m < i 
input RECEIVE(i, j, i) 

eff status := elected 
output SEND(m, i, j) 

pre status # idle A m ©€ pending 


eff pending := delete(m, pending) 
output leader (rank) 

pre status = elected 

eff status := announced 


LCR Leader Election composition automaton 


automaton LCRNode(rank: Int, size: Int) 
components 
P: LCRProcess(rank, size); 
RM[j:Int]: ReceiveMediator (Int, Int, j, rank) 


where j = mod(rank-1, size); 
SM[j:Int]: SendMediator (Int, Int, rank, j) 
where j = mod(rank+1, size) 


Results The trace of a run on 8 nodes (on 4 machines) can be found in Appendix E.1. A snapshot 
of the trace, representing the last five transitions, is shown below. 


A snapshot of the trace of LCR leader election 


transition: output SEND(7, 5, 6) in automaton LCR(5) 
on condor.csail.mit.edu at 7:25:37:280 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 5, 6) in automaton LCR(6) 
on parrot.csail.mit.edu at 7:25:37:755 

Modified state variables: 

P — Tuple, modified fields: {[pending -> (7)] } 


transition: output SEND(7, 6, 7) in automaton LCR(6) 
on parrot.csail.mit.edu at 7:25:37:770 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[7 -> Tuple, modified fields: {[toSend -> 


Sequence, elements added: {7 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 6, 7) in automaton LCR(7) 
on tui.csail.mit.edu at 7:25:37:872 

Modified state variables: 

P — Tuple, modified fields: {[status -> elected] } 


transition: output leader(7) in automaton LCR(7) 
on tui.csail.mit.edu at 7:25:37:874 
Modified state variables: 
P — Tuple, modified fields: {[status -> announced] } 


As the trace indicates, node 7’s message has made its way around the ring and eventually 
returned to node 7. At that point, node 7 announced itself as the leader. Figure 2.1 shows the 
messages sent by the nodes. All messages were sent from node i to node i + 1(mod 8) and the 
message with value 7 was sent first. The message with value 7 followed and made the round of the 
ring, to elect node 7 as the leader. 


Figure 2.1: Running the LCR leader election algorithm on a ring of 8 nodes. The white squares 
represent the nodes, while the shaded parallelograms represent the messages sent. 


2.2 Asynchronous Spanning Tree 


The Asynchronous Spanning Tree Algorithm, (see Section 15.3 of [1]) was the next test for the 
Toolkit. The algorithm is still very simple: Given a general graph it computes a spanning tree on 
the graph. This was the first test of the Toolkit on arbitrary graphs, where each node had more 
than one incoming and outgoing communication channels. 


Automata Definitions The AsynchSpanningTree automaton, as defined in [1](Section 15.3) was 
used. The process automaton and the composition one are listed below. The expanded automaton 
is shown in Appendix C.3. 


Asynchronous Spanning Tree process automaton 


type Message = enumeration of search, null 


automaton sTreeProcess (i: Int) 
signature 
input RECEIVE(m: Message, const i: Int, j: Int) 
output SEND(m: Message, const i: Int, j: Int) 
output PARENT(j: Int) 


states 
nbrs: Set[Int] := {}, 
parent: Int := -1, 4% -1 for null 
reported: Bool := false, 


send: Map[Int, Message] 


transitions 
input RECEIVE(m, i, j) 


eff 
if i#0OA parent = -1 then 
parent := j; 
for k:Int in nbrs - {j} do 
send[k] := search 
od 
fi 
output SEND(m, i, j) 
pre send[j] = search 
eff send[j] := null 


output PARENT(j) 
pre parent = j A reported = false 
eff reported := true 


Asynchronous Spanning Tree composition automaton 


type Message = enumeration of search, null 


automaton sTreeNode (i: Int) 
components 
P: sTreeProcess (i); 
RM[j:Int]: ReceiveMediator (Message, Int, i, j); 
SM[j:Int]: SendMediator (Message, Int, i, j) 


Results Figure 2.2 shows a graph of 16 nodes connected in a 4 x 4 grid that was used on some 
of our tests. The source node was node 0. Some of the spanning trees computed are also shown 
in Figure 2.2. A snapshot of the trace follows, showing nodes 1 and 4 sending their message to 
node 5. The message of node 1 arrives first and thus node 5 announces node 1 as its parent. The 
complete trace of this run can be found in Appendix E.2. 


Trace snapshot of the Asynchronous Spanning Tree algorithm 
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Figure 2.2: The 4 x 4 grid used to test the Asynchronous Spanning Tree Algorithm, along with 2 
different spanning trees computed from 2 different runs of the algorithm. 


transition: output SEND(search, 1, 5) in automaton sTreeNode(1) on 
blackbird.csail.mit.edu 
Modified state variables: 
P — [nbrs: (0 2 5), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 4, 5) in automaton sTreeNode(4) on 
parrot.csail.mit.edu 
Modified state variables: 
P — [nbrs: (0 5 8), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 5, 1) in automaton sTreeNode(5) on 
parrot.csail.mit.edu 
Modified state variables: 
P — [nbrs: (1 4 6 9), parent: 1, reported: false, send: 
ioa.runtime.adt.MapSort] 
RM — ioa.runtime.adt.MapSort 


transition: output PARENT(1) in automaton sTreeNode(5) on 
parrot.csail.mit.edu 
Modified state variables: 
P — [nbrs: (1 4 6 9), parent: 1, reported: true, send: 
ioa.runtime.adt.MapSort] 


2.3. Asynchronous Broadcast Convergecast 


This is essentially an extension of the previous algorithm, where along with the construction of a 
spanning tree, a broadcast and convergecast take place (using the computed spanning tree). The 
source node was node 0, and the message 99 (a dummy message) was broadcast on the network. 
Some complexity is added compared to the previous algorithm by the fact that different kinds of 
messages are exchanged. 


Automata Definitions The AsynchBcastAck automaton, as defined in [1](section 15.3) was 
used. The process automaton and the composition automaton are shown below. The expanded 
automaton is given in Appendix C.4. 


Asynchronous Broadcast Convergecast process automaton 


type Kind = enumeration of bcast, ack 
type BCastMsg = tuple of kind: Kind , w: Int 
type Message = union of msg: BCastMsg, kind: Kind 


automaton bcastProcess (rank: Int, nbrs: Set[Int]) 
signature 
input RECEIVE(m: Message, const rank, j: Int) 
output SEND(m: Message, const rank, j: Int) 
internal report(const rank) 


states 
val: Int := -1, % -1 = special value denoting null 
parent: Int := -i, 
reported: Bool := false, 
acked: Set[Int] := {}, 


send: Map[Int, Seq[Message]] 


initially 
rank = 0> 
(val = 99 A hh 99 = the value to be broadcast 
(V j: Int 


((j € nbrs) => send[j] = {} - msg([bcast, val])))) 


transitions 
output SEND(m, rank, j) 
pre m = head(send[j]) 
eff send[j] := tail(send[j]) 
input RECEIVE(m, rank, j) 
eff 
if m = kind(ack) then 
acked := acked U {j} 
else 
if val = -1 then 
val := m.msg.w; 
parent := j; 
for k:Int in nbrs - {j} do 
send[k] := send[k] | m 
od 
else 


send[j] 


send[j] + kind (ack) 


fi 
fi 
internal report(rank) where rank = 0 
pre acked = nbrs; 
reported = 
eff reported := true 
internal report(rank) where rank # 0 
pre parent # -1; 
acked = nbrs - {parent}; 
reported = false 
send[parent] := 
reported := true; 


false 


eff 


send[parent] | kind(ack); 


Asynchronous Broadcast Convergecast composition automaton 


type Message = union of msg: BCastMsg, kind: Kind 
automaton bcastNode(i:Int) 
components 
P: bcastProcess (i); 
RM[j:Int]: ReceiveMediator (Message, Int, i, j); 
SM[j:Int]: SendMediator (Message, Int, i, j) 
Results The algorithm was tested on several graphs. One of them is shown in Figure 2.3 (a). 


Figure 2.3 (b) and (c) depicts some of the spanning trees that were computed and the communica- 
tion sequence; the numbers next to the nodes represent the sequence at which the nodes reported 
done (through the internal report action). A snapshot of the trace is listed below, showing nodes 
1 and 4 reporting that they are done. At that point, node 0 is enabled and after some communi- 
cation between these nodes, node 0 also reports done. The complete trace of this run is shown in 


Appendix E.3. 


Trace snapshot of the Asynchronous Broadcast Convergecast algorithm 


transition: internal report (1) 

Modified state variables: 

P — [val: 99, acked: (2 5), nbrs: (0 
send: ioa.runtime.adt.MapSort, temp: [kind: 

transition: output SEND(kind(ack), 1, 

Modified state variables: 

P — [val: 99, acked: (2 5), nbrs: (0 
send: ioa.runtime.adt.MapSort, temp: [kind: 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 

Modified state variables: 

P — [val: 99, acked: (5 8), nbrs: (0 
send: ioa.runtime.adt.MapSort, temp: [kind: 


RM — ioa.runtime.adt.MapSort 


transition: internal report (4) 


in automaton bcastNode 


2 5), parent: 0, reported: true, 
beast, w: 87]] 

0) in automaton bcastNode 

2 5), parent: 0, reported: true, 
beast, w: 87]] 


4, 8) in automaton bcastNode 
5 8), parent: 0, reported: false, 
beast, w: 87]] 


in automaton bcastNode 
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Figure 2.3: The grid shown in (a) was used to test the algorithm. (b) and (c) show the spanning 
trees computed in two (different) runs and the numbers next to the nodes indicate the sequence at 
which the nodes reported done. 


Modified state variables: 
P — [val: 99, acked: (5 8), nbrs: (0 5 8), parent: 0, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 4, 0) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5 8), nbrs: (0 5 8), parent: 0, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 0, 1) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (1), nbrs: (1 4), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 99]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 0, 4) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (1 4), nbrs: (1 4), parent: -1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 99]] 

RM — ioa.runtime.adt.MapSort 


transition: internal report(0) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (1 4), nbrs: (1 4), parent: -1, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 99]] 


2.4 Leader Election Using Broadcast Convergecast 


In page 500 of [1], the author describes how the Asynchronous Broadcast Convergecast algorithm 
can be used to implement a leader election algorithm on a general graph using the Asynchronous 
Broadcast Convergecast algortithm. The main idea is to have every node act as a source node 
and create its own spanning tree, broadcast its UID using this spanning tree and hear from all the 
other nodes via a convergecast. During this convergecast, along with the acknowledge message, the 
children also send what they consider as the maximum UID in the network. The parents gather 
the maximum UIDs from the children, compare it to their own UID and send the maximum to 
their own parents. Thus, each source node learns the maximum UID in the network and the node 
whose UID equals the maximum one announces itself as a leader. 


Automata Definitions The process and composition automata are shown below. The expanded 
automaton is defined in Appendix C.5. 


Leader Election Using Broadcast Convergecast process automaton 


type Kind = enumeration of bcast, ack 

type BCastMsg = tuple of kind: Kind, w: Int 

type AckMsg = tuple of kind:Kind, mx: Int 

type MSG = union of bmsg: BCastMsg, amsg: AckMsg, kind: Kind 
type Message = tuple of msg: MSG, source: Int 


automaton bcastLeaderProcess (rank: Int) 
signature 
input RECEIVE(m: Message, i: Int, j: Int) 
output SEND(m: Message, i: Int, j: Int) 
internal report(i: Int, source: Int) 
internal finished 
output LEADER 


states 
nbrs: Set[Intl, 
val: Map[Int, Int], % initialially -1 (null) for all nodes 
parent: Map[Int, Int], % initialially -1 (null) for all nodes 
reported: Map[Int, Bool], % initialially false for all nodes 
acked: Map[Int, Set[Int]], % initialially {} for all nodes 
send: Map[Int, Int, Seq[Message]], % First variable: source. 
max: Map[Int, Int], % The max value found in the network , initially i 
elected: Bool := false, 
announced: Bool := false 

initially 
(VY j: Int 


(O<jAj< 1695 
(rank = j => val[j] = rank 
A rank # j > val[j] = -1 
A parent[j] = -1 
A acked[j] = f} 
A max[j] = rank 
ACV k:Int 
(send[j,k] = {} A 
(k € nbrs A rank = j) > 
send[j, k] = {} - [bmsg([bcast, 99]), j])))) 
transitions 
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output SEND(m, i, j) 
pre m = head(send[m.source, j]) 
eff send[m.source, j] := tail(send[m.source, j]) 
input RECEIVE(m, i, j) 
eff if m.msg = kind(ack) then 
acked[m.source] := acked[m.source] U {j} 
elseif tag(m.msg) = amsg then 
if max[m.source] < m.msg.amsg.mx then 


max[m.source] := m.msg.amsg.mx; 
fi; 
acked[m.source] := acked[m.source] U {j} 
else {BcastMsg 
if val[m.source] = -1 then 
val[m.source] := m.msg.bmsg.w; 
parent[m.source] := j; 
for k:Int in nbrs - {j} do 
send[m.source, k] := send[m.source, k] | m 
od 
else 
send[m.source, j] := send[m.source, j] - [kind(ack), m.source] 
fi 
fi 


internal finished 
pre acked[rank] = nbrs A 


reported[rank] = false 
eff reported[rank] := true; 

if (max[rank] = rank) then 

elected := true 
fi; 
output LEADER 

pre elected = true A announced = false 
eff announced := true 


internal report(i, source) where i # source 
pre parent[source] # -1 A 
acked[source] = nbrs - {parent[source]} A 
reported[source] = false 
eff send[source, parent[source]] := 
send[source, parent[source]] | [amsg([ack, max[source]]), source]; 
reported[source] := true; 


Leader Election Using Broadcast Convergecast composition automaton 


automaton bcastLeaderNode (i: Int) 
components 
P: bcastLeaderProcess (i); 
RM[j:Int]: ReceiveMediator (Message, Int, i, j); 
SM[j:Int]: SendMediator (Message, Int, i, j) 


Results This algorithm was also tested on the 4 x 4 grid that was used in the two previous 
algotithms (see Figure 2.3 (a)). The algorithm terminated correctly and announced node 15 as the 
leader. A snapshot of the trace is shown below. The complete trace of this run can be found on 
the web, at http://theory.csail.mit.edu:/~pmavrom/ioaReport.html 


11 


Trace snapshot of the Leader Election Using Broadcast Convergecast algorithm 


transition: output RECEIVE([msg: amsg([kind: ack, mx: 13]), source: 14], 
14, 10) in automaton bcastLeader (14) on drake.csail.mit.edu at 9:07:20:455 
Modified state variables: 
P — Tuple, modified fields: {[acked -> Map, modified entries: {[14 -> (10 
13 15)]}] } 
SM — Map, modified entries: {[10 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {[msg: amsg([kind: ack, mx: 14]), 
source: 11] }] [sent -> Sequence, elements added: {[msg: amsg([kind: ack, mx: 
14]), source: 11] } Elements removed: {}] }]} 
c2 — 7 


transition: internal finished() in automaton bcastLeader (14) 
on drake.csail.mit.edu at 9:07:20:464 
Modified state variables: 
P — Tuple, modified fields: {[reported -> Map, modified entries: {[14 -> 


true]}] } 
c2 — 14 
k — 15 


transition: output RECEIVE([msg: amsg([kind: ack, mx: 14]), source: 15], 
15, 11) in automaton bcastLeader (15) on loon.csail.mit.edu at 9:07:20:684 
Modified state variables: 
P — Tuple, modified fields: {[acked -> Map, modified entries: {[15 -> (11 
14)]}] } 
SM — Map, modified entries: {[11 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {[msg: amsg([kind: ack, mx: 15]), 
source: 4] }] [sent -> Sequence, elements added: {[msg: amsg([kind: ack, mx: 
15]), source: 4] } Elements removed: {}] }]} 
c2 > 7 


transition: internal finished() in automaton bcastLeader (15) 
on loon.csail.mit.edu at 9:07:20:687 

Modified state variables: 

P — Tuple, modified fields: {[reported -> Map, modified entries: {[15 -> 
true]}] [elected -> true] } 

c2 — 15 


transition: output LEADER() in automaton bcastLeader (15) 
on loon.csail.mit.edu at 9:07:20:689 

Modified state variables: 

P — Tuple, modified fields: {[announced -> true] } 


2.5 Unrooted Spanning Tree to Leader Election 


The algorithm STtoLeader of [1](page 501) was implemented as the next test for the Toolkit. The 
algorithm takes as input an unrooted spanning tree and returns a leader. The automaton listed 
below was written, according to the description of the algorithm. 


Automata Definitions The process and composition automata are listed below, while the ex- 
panded automaton can be found in Appendix C.6. 
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Unrooted Spanning Tree to Leader Election process automaton 


type Status = enumeration of idle, elected, announced 
type Message = enumeration of elect 
axioms ChoiceSet (Int for E) 


automaton sTreeLeaderProcess (rank: Int, nbrs:Set[Int]) 
signature 
input RECEIVE(m: Message, const rank: Int, j: Int) 
output SEND(m: Message, const rank: Int, j: Int) 
output leader 
states 
receivedElect: Set[Int]l, 
sentElect: Set[Int], 
status: Status, 
send: Map[Int, Seq[Message]] 
initially 
true > 
receivedElect = {} 
A sentElect = {} 
A status = idle 
A (size(nbrs) = 15> 
send [chooseRandom (nbrs) ] 
= send[chooseRandom(nbrs)] - elect) 
transitions 
input RECEIVE(m, i, j) 
eff receivedElect := insert(j, receivedElect ); 
if size(receivedElect) = size(nbrs) - 1 then 
send[chooseRandom(nbrs - receivedElect)] := 
send[chooseRandom(nbrs - receivedElect)] | elect; 
sentElect := 
insert (chooseRandom(nbrs - receivedElect), sentElect) 
elseif receivedElect = nbrs then 
if j € sentElect then if i> j then status := elected fi 
else status := elected 
fi 
fi 
output SEND(m, i, j) 
pre m = head(send[j]) 


eff send[j] := tail(send[j]) 
output leader 

pre status = elected 

eff status := announced 


Unrooted Spanning Tree to Leader Election composition automaton 


automaton sTreeLeaderNode (rank:Int, nbrs:Set[Int]) 
components 
P: sTreeLeaderProcess (rank, nbrs); 
RM[j:Int]: ReceiveMediator (Message, Int, rank, j); 
SM[j:Int]: SendMediator (Message, Int, rank, j) 


Results The algorithm was tested on several graphs and different spanning trees as input. A 
spanning tree on a 4 x 4 grid is shown in Figure 2.4. The algorithm worked correctly, announcing 
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only one node as the leader. A snapshot of the trace is listed below. In that specific run, the edge 
between nodes 5 and 6 had elect messages sent in both directions, with node 6 being elected as the 
leader, having a larger UID. The complete trace of this run can be found in Appendix E.4. 


0 1 2 3 0 >> 1 Qt 1'3 
| A 

Y 
4 5 6 7 4, ~~ 5 *— 6 7 


t A A 
8 9 10 11 8 r+ 9 | 10 11 
A A 


12 13 14 15 12 13 ;—> 14 15 


Figure 2.4: (a) shows the unrooted spanning tree used in one run. In (b), the arrows show the 
direction of the messages in a specific run, where the edge between nodes 5 and 9 had elect messages 
sent in both directions. Node 9, with a larger UID was elected as the leader. 


Trace Snapshot of the Unrooted Spanning Tree to Leader Election 


transition: output SEND(elect, 6, 5) in automaton sTreeLeader (6) 

on drake.csail.mit.edu at 9:29:15:112 

Modified state variables: 

P — Tuple, modified fields: {[send -> Map, modified entries: {[5 -> 
Sequence, elements added: {} Elements removed: felect }]}] } 

SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: felect } Elements removed: {}] }]} 

k > 5 

tempNbrs2 — (10 2) 


transition: output SEND(elect, 5, 6) in automaton sTreeLeader (5) 

on loon.csail.mit.edu at 9:29:16:182 

Modified state variables: 

P — Tuple, modified fields: {[send -> Map, modified entries: {[6 -> 
Sequence, elements added: {} Elements removed: felect }]}] } 

SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: felect } Elements removed: {}] }]} 

k — 6 

tempNbrs2 — (1 4 9) 


transition: output RECEIVE(elect, 5, 6) in automaton sTreeLeader (5) 

on loon.csail.mit.edu at 9:29:16:494 

Modified state variables: 

P — Tuple, modified fields: {[receivedElect -> (1 4 6 9)] } 

SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: felect }] 
[sent -> Sequence, elements added: felect } Elements removed: {}] }]} 


14 


transition: output RECEIVE(elect, 6, 5) in automaton sTreeLeader (6) 
on drake.csail.mit.edu at 9:29:16:554 

Modified state variables: 

P — Tuple, modified fields: {[receivedElect -> (10 2 5)] [status -> 
elected] } 

SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: felect }] [sent -> 
Sequence, elements added: felect } Elements removed: {}] }]} 


transition: output leader() in automaton sTreeLeader (6) 
on drake.csail.mit.edu at 9:29:16:559 

Modified state variables: 

P — Tuple, modified fields: {[status -> announced] } 


2.6 GHS Minimum Spanning Tree 


The last algorithm we ran using the Toolkit was the algorithm of Gallager, Humblet and Spira for 
finding the minimum-weight spanning tree in an arbitrary graph. Welch, Lamport and Lynch give 
an I/O automaton definition of the GHS algorithm in [2], which is used as a basis for our automata 
definitions. One technical detail in running this version of the algorithm is that the edge weights 
needed to be unique. This algorithm is significantly longer than the previous ones, with 7 different 
messages exchanged and 12 state variables for each automaton. 


Automata Definitions The automaton below defines the algorithm and the composition with 
the mediator automata is shown after that. The expanded automaton can be found in Ap- 
pendix C.7. 


GHS process automaton 


type Nstatus = enumeration of sleeping, find, found 

type Edge = tuple of s: Int, t: Int 

type Link = tuple of s: Int, t: Int 

type Lstatus = enumeration of unknown, branch, rejected 

% Message Types 

type Msg = enumeration of CONNECT, INITIATE, TEST, REPORT, ACCEPT, 
REJECT, CHANGEROOT 

type ConnMsg = tuple of msg: Msg, 1: Int 

type Status = enumeration of find, found 

type InitMsg = tuple of msg: Msg, 1: Int, c: Null[Edge], st: Status 

type TestMsg = tuple of msg: Msg, 1: Int, c: Null [Edge] 

type ReportMsg = tuple of msg: Msg, w: Int 


type Message = union of connMsg: ConnMsg, initMsg: InitMsg, 
testMsg: TestMsg, reportMsg: ReportMsg, 
msg: Msg 


uses ChoiceSet (Link) 


% automaton GHSProcess: Process of GHS Algorithm for min. spanning tree 
% rank: The UID of the automaton, automatically initialized by MPI 

% size: The size of the network, automatically initialized by MPI 

% links: Set of Links with source = rank (L_p(G)) 
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% weight: Maps the Links € links to their weight 
hh 
automaton GHSProcess(rank: Int, size: Int, links: Set[Link], weight: Map[Link, Int]) 
signature 
input startP 
input RECEIVE(m: Message, const rank, i: Int) 
output InTree(1:Link) 
output NotInTree(1l: Link) 
output SEND(m: Message, const rank, j: Int) 
internal ReceiveConnect (qp: Link, 1:Int) 
internal ReceiveInitiate (qp: Link, 1:Int, c: Null[Edge], st: Status) 
internal ReceiveTest (qp: Link, 1:Int, c: Null [Edge]) 
internal ReceiveAccept (qp: Link) 
internal ReceiveReject (qp: Link) 
internal ReceiveReport (qp: Link, w: Int) 
internal ReceiveChangeRoot (qp: Link) 


states 
nstatus: Nstatus, 
nfrag: Null[Edge]l, 
nlevel: Int, 
bestlink: Null [Link], 
bestwt: Int, 
testlink: Null [Link], 
inbranch: Link, 
findcount: Int, 
lstatus: Map[Link, Lstatus], 
queueQOut: Map[Link, Seq[Message]], 
queueIn: Map[Link, Seq[Message]], 
answered: Map[Link, Bool], 
% Temporary variables 
min: Int, minL: Null[Link], S: Set [Link] 


initially 
true > nstatus = sleeping 
A nfrag = nil 
A nlevel = 0 
A bestlink = embed(chooseRandom (links) ) 
A bestwt = weight [chooseRandom (links) ] 
A testlink = nil 
A inbranch = chooseRandom (links) 
A findcount = 0 
A V1: Link 
(1 € links > 
lstatus[1] = unknown 
A answered[1] = false 


A queueOut[1] = {} 
A queueIn[1] = {}) 
transitions 
% INPUT ACTIONS 
input startP 
eff if nstatus = sleeping then 


“4WakeUp 
minL := embed(chooseRandom(links)); min := weight[minL.val]; 
for tempL:Link in links do 
if weight[tempL] < min then minL := embed(tempL); min := weight[tempL] fi; 
od; 


16 


lstatus[minL.val] := branch; nstatus := found; 
queueOut [minL.val] := queueOut [minL.val] - connMsg([CONNECT, 0]); 
Z4EndWakeUp 
fi 
input RECEIVE(m: Message, i:Int, j:Int) 
eff queueIn[[i,j]] := queuveIn[[i,j]] } m 
% OUTPUT ACTIONS 
output InTree(1l: Link) 
pre answered[1l] = false A lstatus[1l] = branch 


eff answered[1l] := true 

output NotInTree(1l: Link) 
pre answered[1] = false A lstatus[1] = rejected 
eff answered[1l] := true 


output SEND(m: Message, i: Int, j: Int) 
pre m = head(queueOut [[i,j]]) 
eff queueOut[[i,j]] := tail (queueOut [[i,j]]) 
% INTERNAL ACTIONS 
internal ReceiveConnect (qp: Link, 1: Int) 
pre head(queueIn[qp]) = connMsg([CONNECT, 1]) 
eff queueIn[gqp] := tail(queueIn[qp]); 
if nstatus = sleeping then 
4WakeUp 
minL := embed(chooseRandom(links)); min := weight[minL.val]; 
for tempL:Link in links do 
if weight[tempL] < min then minL := embed(tempL); min := weight [tempL] 
od; 
lstatus[minL.val] := branch; nstatus := found; 
queueOut [minL.val] := queueOut [minL.val] + connMsg([CONNECT, 0]); 
ZEndWakeUp 
fi; 
if 1 < nlevel then 
lstatus[[qp.t,qp.s]] := branch; 
if testlink # nil then 
queueOut [[qp.t,qp.s]] := queueOut[[qp.t,qp.s]] | initMsg([INITIATE, 
nlevel, nfrag, find]); 
findcount := findcount + 1 
else 
queueOut [[qp.t,qp.s]] := queueOut[[qp.t,qp.s]] + initMsg([INITIATE, 
nlevel, nfrag, found]) 
fi; 
else 
if lstatus[[qp.t,qp.s]] = unknown then 
queueIn[qp] := queueIn[qp] - connMsg([CONNECT, 1]) 
else 
queueOut [[qp.t,qp.s]] := queueOut[[qp.t,qp.s]] + initMsg([INITIATE, 
nlevel+i, embed([qp.t, qp.s]), find]) 
fi 
fi 
internal ReceiveInitiate (qp: Link, 1:Int, c: Null[Edge], st: Status) 
pre head(queuveIn[qp]) = initMsg([INITIATE, 1, c, st]) 


eff queuveIn[qp] := tail(queueIn[qp]); 
nlevel := 1; 
nfrag := Cc; 
if st = find then nstatus := find else nstatus := found fi; 
% - Let S = { [p,q] : lstatus[[p,r]] = branch, r 4 q} - 
Ss := {}; 
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fi; 


for pr: Link in links do 
if pr.t # qp.s A lstatus[pr] = branch then 


S := S U {pr} 

fi 
od; 
for k: Link in S do 

queueQOut[k] := queuveOut[k] + initMsg([INITIATE, 1, c, st]) 
od; 
if st = find then 

inbranch := [qp.t, qp.s]; 

bestlink := nil; 

bestwt := 10000000; % Infinity 

“%TestP 

minL := nil; min := 10000000; % Infinity 


for tempL:Link in links do 
if weight[tempL] < min A lstatus[tempL] = unknown then 


minL := embed(tempL); min := weight [tempL] 
fi; 
od; 
if minL # nil then 
testlink := minL; 
queueQut [minL.val] := queueOut[minL.val] + testMsg([TEST, nlevel, nfrag]); 
else 
testlink := nil; 
ZReport 
if findcount = 0 A testlink = nil then 
nstatus := found; 
queueQut [inbranch] := queueOut[inbranch] + reportMsg([REPORT, bestwt]) 
fi 
ZEndReport 
fi; 
%ZEndTestP 
findcount := size(S) 


fi 


internal ReceiveTest(qp: Link, 1: Int, c: Null [Edge]) 
pre head(queueIn[qp]) = testMsg([TEST, 1, c]) 


eff queueIn[gqp] := tail(queueIn[qp]); 
if nstatus = sleeping then 
4WakeUp 
minL := embed(chooseRandom(links)); min := weight[minL.val]; 
for tempL:Link in links do 
if weight[tempL] < min then minL := embed(tempL); min := weight[tempL] fi; 
od; 
lstatus[minL.val] := branch; nstatus := found; 
queueOut [minL.val] := queueOut [minL.val] - connMsg([CONNECT, 0]); 
ZEndWakeUp 
fi; 
if 1 > nlevel then 
queueIn[qp] := queueIn[qp] | testMsg([TEST, 1, c]); 
else 
if c # nfrag then 
queueOut[[qp.t, qp.s]] := queuveOut[[qp.t, qp.s]] | msg(ACCEPT) 
else 
if lstatus[[qp.t, qp.s]] = unknown then lstatus[[qp.t, qp.s]] := rejected fi; 


if testlink # embed([qp.t, qp.s]) then 
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queueOut[[qp.t, gqp.s]] := queveOut[[qp.t, qp.s]] | msg(REJECT) 
else 


ATest 
minL := nil; min := 10000000; % Infinity 
for tempL:Link in links do 
if weight[tempL] < min A lstatus[tempL] = unknown then 
minL := embed(tempL); min := weight[tempL] 
fi; 
od; 
if minL # nil then 
testlink := minL; 
queueQut [minL.val] := queueOut[minL.val] - testMsg([TEST, nlevel, nfrag]); 
else 
testlink := nil; 
“Report 
if findcount = 0 A testlink = nil then 
nstatus := found; 
queueQut [inbranch] := queueOut [inbranch] + reportMsg([REPORT, bestwt]) 
fi 
ZEndReport 
fi; 
ZEndTest 
fi; 
fi; 
fi; 


internal ReceiveAccept (qp: Link) 
pre head(queueIn[qp]) = msg (ACCEPT) 


eff queueIn[gqp] := tail(queueIn[qp]); 

testlink := nil; 

if weight[[qp.t, qp.s]] < bestwt then 
bestlink := embed([qp.t, qp.s]); 
bestwt := weight[[qp.t, gp.s]]; 

fi; 

“Report 

if findcount = 0 A testlink = nil then 
nstatus := found; 
queueOut[inbranch] := queueQut[inbranch] + reportMsg([REPORT, bestwt]) 

fi 

ZEndReport 


internal ReceiveReject (qp: Link) 
pre head(queueIn[qp]) = msg (REJECT) 


eff queueIn[gp] := tail(queueIn[qp]); 
if lstatus[[qp.t, qp.s]] = unknown then l1status[[qp.t, qp.s]] := rejected fi; 
“Test 
minL := nil; min := 10000000; % Infinity 


for tempL:Link in links do 
if weight[tempL] < min A lstatus[tempL] = unknown then 


minL := embed(tempL); min := weight[tempL] 
fi; 
od; 
if minL # nil then 
testlink := minL; 
queueOut [minL.val] := queueOut [minL.val] + testMsg([TEST, nlevel, nfrag]); 
else 
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testlink := nil; 


“Report 
if findcount = 0 A testlink = nil then 
nstatus := found; 
queueQut [inbranch] := queueOut[inbranch] + reportMsg([REPORT, bestwt]) 
fi 
ZEndReport 
fi 
ZEndTest 


internal ReceiveReport (qp: Link, w: Int) 
pre head(queueIn[qp]) = reportMsg([REPORT, w]) 


eff queueIn[gp] := tail(queueIn[qp]); 
if [qp.t, qp.s] # inbranch then 
findcount := findcount -1; 
if w < bestwt then 
bestwt := WwW; 
bestlink := embed([qp.t, qp.s]) 
fi; 
“Report 
if findcount = 0 A testlink = nil then 
nstatus := found; 
queueOut[inbranch] := queueOut[inbranch] + reportMsg([REPORT, bestwt]) 
fi 
ZEndReport 
else 
if nstatus = find then 
queueIn[qp] := queueIn[qp] + reportMsg([REPORT, w]) 
elseif w > bestwt then 
4ChangeRoot 
if lstatus[bestlink.val] = branch then 
queueOut [bestlink.val] := queueOut [bestlink.val] - msg (CHANGEROOT ) 
else 
queueOut [bestlink.val] := queueOut [bestlink.val] + 
connMsg([CONNECT, nlevel]) ; 
lstatus [bestlink.val] := branch 
fi 
Z4EndChangeRoot 
fi 
fi 


internal ReceiveChangeRoot (qp: Link) 
pre head(queueIn[qp]) = msg (CHANGEROOT ) 


eff queveIn[qp] := tail(queueIn[qp]); 

4ChangeRoot 

if lstatus[bestlink.val] = branch then 
queueOut [bestlink.val] := queueOut [bestlink.val] - msg (CHANGEROOT ) 

else 
queueQut [bestlink.val] := queueOut [bestlink.val] - connMsg([CONNECT, nlevel]) ; 
lstatus[bestlink.val] := branch 

fi 

4EndChangeRoot 


GHS composition automaton 
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automaton GHSNode(rank: Int, size: Int, links: Set[Link], weight: Map[Link, Int]) 
components 
P: GHSProcess(rank, size, links, weight); 
RM[j:Int]: ReceiveMediator (Message, Int, rank, j); 
SM[j:Int]: SendMediator (Message, Int, rank, j) 


Results One of the graphs that were used to test the algorithm is shown in Figure 2.5 (a). The 
(unique) minimum spanning tree computed by the algorithm is shown in Figure 2.5 (b). A snap- 
shot of the trace follows, showing the beginning of the algorithm with node 0 waking up (after a 
start action), sending a CONNECT message to its minimun weight outgoing edge (1) and reporting 
this edge as part of the minimum spanning tree. The complete trace of this run can be found 
at http://theory.csail.mit.edu:/~pmavrom/ioaReport.html. The algorithm ran correctly, re- 
turning the unique minimum spanning tree in all cases. 
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Figure 2.5: (a) shows one of the graphs used to test the GHS algorithm. In (b), the (unique) 
minimum spanning tree computed by the algorithm 


Trace snapshot of the GHS algorithm 
Initialization starts (0) on loon.csail.mit.edu at 7:05:55:830 


Modified state variables: 
P — [nstatus: sleeping, nfrag: nil, nlevel: 87, bestlink: nil, bestwt: 
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87, testlink: nil, inbranch: [s: 87, t: 87], findcount: 87, lstatus: Map{}, 
queueOut: Map{}, queueIn: Map{}, answered: Map{}, min: 87, minL: nil, S: ()] 

RM — Map{} 

SM — Map{} 

links — (Cs: 0, t: 1] [s: 0, t: 4]) 

ink — [s: 87, t: 87] 

Inks — () 

rank — null 

size — null 

tempL — [s: 87, t: 87] 

tempLinks — () 

weight — Map{[[s: 0, t: 1] -> 8] [[s: 0, t: 4] -> 12] } 

Initialization ends 

transition: input startP() in automaton GHS (0) 
on loon.csail.mit.edu at 7:05:55:852 

Modified state variables: 

P — [nstatus: found, nfrag: nil, nlevel: 0, bestlink: embed([s: 0, t: 
4]), bestwt: 12, testlink: nil, inbranch: [s: 0, t: 1], findcount: 0, lstatus: 
Map{[[s: 0, t: 1] -> branch] [[s: 0, t: 4] -> unknown] }, queueOut: Map{[[s: 0, 
t: 1] -> {connMsg([msg: CONNECT, 1: 0])}] [[s: 0, t: 4] -> {}] }, queueIn: 
Map{[[s: 1, t: 0] -> {}] [E[s: 4, t: 0] -> {}] }, answered: Map{[[s: 0, t: 1] -> 
false] [[s: 0, t: 4] -> false] }, min: 8, minL: embed([s: 0, t: 1]), S: Q] 
RM — Map{[1 -> [status: idle, toRecv: {}, ready: false]] [4 -> [status: 
idle, toRecv: {}, ready: false]] } 

SM — Map{[1 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [4 -> 
[status: idle, toSend: {}, sent: {}, handles: {}]] } 

links — (Cs: 0, t: 1] [s: 0, t: 4]) 

ink — [s: 87, t: 87] 

Inks — () 

rank — 0 

size — 16 

tempL — [s: 0, t: 4] 

tempLinks — () 

weight — Map{[[s: 0, t: 1] -> 8] [[s: 0, t: 4] -> 12] } 


transition: output SEND(connMsg([msg: CONNECT, 1: 0]), 0, 1) in automaton 
GHS (0) on loon.csail.mit.edu at 7:05:55:864 

Modified state variables: 

P — Tuple, modified fields: {[queueOut -> Map, modified entries: {[[s: 0, 
t: 1] -> Sequence, elements added: {} Elements removed: {connMsg([msg: CONNECT, 
1: 0]) }]}] } 

SM — Map, modified entries: {[1 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {connMsg([msg: CONNECT, 1: 0]) } Elements removed: {}] 
$1} 

lnk — Tuple, modified fields: {[s -> 0] [t -> 1] } 


transition: output InTree([s: 0, t: 1]) in automaton GHS (0) 
on loon.csail.mit.edu at 7:05:55:915 

Modified state variables: 

P — Tuple, modified fields: {[answered -> Map, modified entries: {[[s: 0, 
t: 1] -> true]}] } 

SM — Map, modified entries: {[1 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {connMsg([msg: CONNECT, 1: 0]) }] 
[sent -> Sequence, elements added: {connMsg([msg: CONNECT, 1: 0]) } Elements 
removed: {}] }]} 
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3 Results 


Figures 3.6, 3.7 and 3.8 depict the runtime results of the algorithms on 1 node per machine, a 
constant number of nodes and a constant number of machines respectively. The runtimes were 
measured in seconds, from the time when the first node started initialization until the time at 
which the algorithm terminated, and they are averages on 10 runs. The tables in Appendix D show 
the raw data collected from our experiments. In Figure 3.6(a) to (b) the relationship between time 
and number of nodes is not very clear due to the short time taken, number of nodes and messages 
exchanged. Figure 3.6(d) however, where the number of messages exchanged is large shows the 
clear (linear) relation. 


4 Conclusions 


During the months of June and July 2004 we had the opportunity to test the capabilities and the 
performance of the version of the toolkit that was then available. We started by mostly modifying 
the Java code the Toolkit generated to get LCR up and running. The toolkit was then gradually 
modified to automate the manual changes we had to make in Java. By the Spanning Tree to 
Leader algorithm the Toolkit was completely automated to provide Java code that compiled and 
ran successfully. After that, and after making sure that simple algorithms such as a simple broadcast 
and convergecast could be run, we implemented a more complicated algorithm, the GHS algorithm 
for computing the minimum spanning tree in an arbitrary graph. The successfull implementation 
of this algorithm makes us very confident that the toolkit can be used to implement complex 
distributed systems successfully. 


Performance We now comment on some performance issues: 


1. Scalability: As Figure 3.6 suggests, the Toolkit is scalable. Letting the number of nodes double 
increases the runtime but no more than twice the previous runtime. Moreover, Figure 3.6(e) 
shows that the rate of increase of the running time is smaller as the number of messages 
increases. 


2. Setup time: The time required for MPI to set up all the connections and enable the nodes to 
initialize was not measured in the runtime results. However, when the number of nodes was 
large, this time was also quite significant (around 5-10 minutes). 


3. Resource usage: The generated programs used all the available processing power available to 
them during the whole run. This was mainly because there was no pause between suceessive 
tests for incoming or outgoing messages. 


5 Comments and Suggestions on the Toolkit 


The NDR Language, used both in the scheduler and the initialization block could be extended 
to support the for v:T in s:Set[T] statement, which is already used in IOA. This would 
make the code much cleaner since using the current syntax, these loops are implemented using 
a while loop and an extra set of two or more variables. 
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Figure 3.6: (a) to (d) show the time taken to complete the algorithm in regard to the number 
of nodes. (e) shows the time of Broadcast Leader Election in regard to the number of messages 
exchanged. 
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Figure 3.7: (a) to (c) show the time taken to terminate, in regard to the number of physical 
machines used. 
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Figure 3.8: (a) to (c) show the time taken to terminate, in regard to the number of nodes. 
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e Initialization of Map (or Array) entries. The toolkit allows usage of possibly non-initialized 
map/ array entries. As a result, when the compiled code is run, a NullPointerException is 
thrown from the Java environment when these variables are accessed. The following example 


illustrates the problem and demonstrates some possible solutions: 
type Tup = tuple of a:Int, b: Bool 

automaton Test 

signature 


states 
M: Map[Tup] 
initially 
(true > SM[1].a = 3) 
det do 

SM[i]J.a := 3 


The user here tries to initialize the tuple before constructing it, which is done using a com- 
mand like sM[1] := [3, true]. One possible solution would be for the toolkit to check for an 
initialization statement of the type sM[1] := ... in the initially block, before the accessing 
statement (of the type sM[1].a). If none exists, it could print a warning. Another solution 
could be to arbitrarily initialize all Map / Array Entries. The toolkit already supports such 
kind of initialization, but again if it does not occur, a warning message could be printed. 


e In GHS, some parts of the IOA code were used more than once. The code given in [2] makes 
use of a procedure, i.e. a block of statements that can be declared and then called as many 
times as necessary. Given the complexity of GHS, such a feature would be quite useful during 
the design, coding and debugging process. For GHS, there was no need for parameter passing 
to the procedure. Therefore, as a first step, a simple procedure support (without parameter 
passing) from the toolkit would be enough. 


e The use of MPI for communication has both advantages and disadvantages. Implementing 
the Toolkit was probably easier using MPI. No low-level communication programming was 
necessary. Furthermore, it has been tested to work for a long time now, and indeed, it works. 
However, it introduces some issues that could have been avoided. Firstly, most of the error 
messages coming from MPI are far from descriptive. (see Troubleshooting # 4) Moreover, MPI 
sets up a connection between all pairs of nodes, even if these connections are not necessary. 
An n-node LCR needs only n connections, while MPI sets up ©(n?) connections. This is 
probably the reason why on a larger number of nodes, MPI takes up a lot of time to setup. 
We suggest that the possibility of another communication interface, which gives more control 
over these issues (e.g. Java RMI) is examined at some time in the future. 


e Finally, we have tested the possibility of pausing between consecutive tests for incoming and 
outgoing messages (MPI’s test and Iprobe). Right now a node might be in an infinite loop 
probing for messages for tens of seconds and using up all the processing power available to 
it. Forcing the nodes to sleep for some milliseconds before probing again showed to improve 
performance when the number of nodes per physical machine is large. We have experimented 
with some runtimes of LCR Leader election on 20 nodes on a single machine, using different 
sleep times. The results are shown below: 
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Sleep time (milliseconds) | LCR on 20 nodes, 1 machine runtime (seconds) 
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Troubleshooting 


. Sometimes during the composition of the process and mediator automata, two SEND and/or 
two RECEIVE transitions were produced by the composer. This should be fixed before 
generating Java code because the code will not compile later. To prevent this one should make 
sure that the where clauses on the SEND and RECEIVE actions in the process automaton 
match the where clauses for the SendMediator and ReceiveMediator component automata in 
the node composition. 


. Send/ Receive Convention. In LCR, we have used the convention of [1] for the Send and 
Receive transitions: Send(m, i, j) and Receive(m, j, i). This means that i is always 
the sender and j is always the receiver. This convention sometimes caused problems that were 
hard to debug. We found that using a different convention made it easier to understand and 
debug any problems: Send(m, i, j) and Receive(m, i, j), meaning that node i sends to 
node j and node i receives from node j respectively. 


. Currenlty the NDR Language, used in the schedule and initialially blocks does not support 
the for v:T in S:Set(T]. The way to create such loops is to use a while loop of the form: 


uses ChoiseSet 


schedule 


states 
tempNbrs: Set [Int], 
k: Int 
do 
tempNbrs := P.nbrs; 
while (7isEmpty (tempNbrs)) do 
k := chooseRandom(tempNbrs) ; 
tempNbrs := delete(k, tempNbrs); 
\% fire action on neighbor k 
od; 
od 


. MPI throws a SIGSEV error if the program tries to access a node using a negative number 
as the rank. 


. Several times when running an algorithm we encountered a java. lang.NullPointerException 
during the initialization or the first transitions of the automata. Almost always, this was due 
to a wrong initialization of the state variables in IOA. 


Bug Reports 


. In the LCRNode automaton some types such as Status and _States[LCRProcess] were not 
declared automatically. Manual declaration of these types was necessary. The latest version 
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of the toolkit generates these type definitions automatically. [Fixed] 


. The transitions that connect to the MPI have different signatures, but the implementation of 
the toolkit did not support that. (e.g. Iprobe(i_v1,j_v2) and resp_Iprobe(i_v3, j_v4) 
were not recognized as an MPI pair). We manually changed them to have exactly the same 
signature, for example Iprobe(i_v1, j_v2) and resp_Iprobe(i_v1, j_v2). The latest 
version of the toolkit recognizes them as pairs even if they have different names. [Fixed] 


. In LCR.a statement (const 0) was produced that was not recognized by the code generator. 
We manually changed the IL file produced to (const v999), and declared the variable v999 to 
something like (v999 zero s8 (scope 1)) where s8 was the sort corresponding to NatSort. 
[Pending] 


. Formal parameters were not translated in Java. We manually declared the field in the gen- 
erated code, and initialized it to the correct value. The toolkit now translates all formal 
parameters and automatically initializes three special parameters: rank, size (of type Int or 
Nat) and hostName (of type String). [Fixed] 


. The initialization of the arguments from MPI (the statement MPI. Init (args) ;) had to hap- 
pen in the main() method of the class LCRNode and not ioa.runitime.Automaton. Otherwise, 
the environment would not recognize the correct number of processes running. The toolkit 
has been modified so that this will happen automatically from now on. [Fixed] 


. The composer tool (incorrectly) removes all the preconditions of internal actions during com- 
position. [Pending] 


. In the NDR Language translation, no break; statement was produced after a fire block. This 
caused unexpected termination in some algorithms. [Fixed] 


. IntSort’s modulo operator is not compatible with its specification. [Pending] 


. The syntax of declaring a map is v: Map[D1i, ..., Dn, R] where n > 1. If only one type is 
given, (e.g. map1: Map[Int]), the checker tool throws a java.lang.InternalError instead of 
a more descriptive Syntax Error message. 
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C Automata Definitions and Input Files 


C.1 SendMediator and ReceiveMediator 


These automata were used as the channel between two nodes and the connection of the automata 
to the MPI. 


SendMediator 


type sCall = enumeration of idle, Isend, test 
automaton SendMediator (Msg, Node:Type, i:Node, j:Node) 
assumes Infinite (Handle) 


signature 
input SEND(m: Msg, const i, const j) 
output Isend(m: Msg, const i, const j) 
input resp_Isend(handle:Handle, const i, const j) 
output test (handle:Handle, const i, const j) 
input resp_test(flag:Bool, const i, const j) 


states 
status: sCall := idle, 
toSend: Seq[Msg] := {}, 
sent: Seq[Msg] := {}, 
handles: Seq[Handle] := {} 


transitions 
input SEND(m, i, j) 
eff toSend := toSend - m 
output Isend(m,i,j) 
pre head(toSend) = mn; 


status = idle 
eff toSend := tail(toSend); 
sent := sent} n; 
status := Isend 
input resp_Isend(handle, i, j) 
eff handles := handles | handle; 
status := idle 
output test(handle, i, j) 
pre status = idle; 
handle = head (handles) 
eff status := test 


input resp_test(flag, i, j) 
eff if (flag = true) then 


handles := tail(handles); 
sent := tail(sent) 
fi; 
status := idle 
ReceiveMediator 
type rCall = enumeration of idle, receive, Iprobe 


3l 


automaton ReceiveMediator (Msg, Node: Type, i: Node, 


assumes Infinite (Handle) 


signature 
output RECEIVE(m:Msg, const i, const j) 
output Iprobe(const i, const j) 
input resp_Iprobe(flag:Bool, const i, const j) 
output receive(const i, const j) 
input resp_receive(m: Msg, const i, const j) 


states 
status: rCall := idle, 
toRecv: Seq[Msg] := {}, 
ready: Bool := false 


transitions 
output RECEIVE(m, i, j) 
pre m = head(toRecv) 


eff toRecv := tail(toRecv) 
output Iprobe(i, j) 
pre status = idle; 
ready = false 
eff status := Iprobe 
input resp_Iprobe(flag, i, j) 
eff ready := flag; 
status := idle 
output receive(i, j) 
pre ready = true; 
status = idle 
eff status := receive 
input resp_receive(m, i, j) 
eff toRecv := toRecv | m; 
ready := false; 
status := idle 
C.2 LCR 


j: Node) 


Expanded automaton with initialization and scheduling 


uses ChoiceSet (Int) 


automaton LCR(rank: Int, size: Int) 
signature 
output receive(N6: Int, N7: Int) 


where N7 = rank A N6 = mod((rank+size) -1, size) 


output SEND(m: Int, I2: Int, I3: Int) 
where I3 = mod(rank + 1, size) A I2 = rank 
input resp_Iprobe(flag: Bool, N4: Int, N5: Int) 


where N5 = rank A N4 = mod((rank+size) -1, size) 


input resp_test(flag: Bool, N18: Int, N19: Int) 
where N19 = mod(rank + 1, size) A N18 = rank 
input resp_receive(m: Int, N8: Int, N9: Int) 


where N9 = rank A N8 = mod((rank+size) -1, size) 


input vote 
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output test (handle: Handle, N16: Int, N17: Int) 
where N17 = mod(rank + 1, size) A N16 = rank 
output Iprobe(N2: Int, N3: Int) 


where N3 = rank A N2 = mod((rank+size) -1, size) 
output RECEIVE(m: Int, I0: Int, I1: Int) 
where Ii = rank A IO = mod((rank+size) -1, size) 


output leader(I4: Int) where I4 = rank 
input resp_Isend(handle: Handle, N14: Int, N15: Int) 
where N15 = mod(rank + 1, size) A N14 = rank 
output Isend(m: Int, N12: Int, N13: Int) 
where N13 = mod(rank + 1, size) A Ni2 = rank 
states 
P: _States[LCRProcess], 
RM: Map[Int, _States[ReceiveMediator, Int, Int]], 
SM: Map[Int, _States[SendMediator, Int, Int]] 


initially 
(true > P.pending = {rank} A P.status = idle) 
/\ 
Voj: Int 
((j = mod((rank+size) -1, size) 
=> 
RM[j].status = idle 
A RM[j].toRecv = {} 
A RM[j].ready = false) 
A (j = mod((rank+size) -1, size) <= defined(RM, j))) 
A 
Voj: Int 
((j = mod(rank + 1, size) 
=> 
SM[j].status = idle 
A SM[j].toSend = {} 
A SMEj].sent = {} 
A SM[j].handles = {}) 
A (j = mod(rank + 1, size) = defined(SM, j))) 
det do 
P.pending := {rank}; 
P.status := idle; 
RM[mod((rank+size) -1, size)] := [idle, {}, false]; 
SM[mod(rank+1i, size)] := [idle, {}, {}, {}] 
od 


transitions 
output receive(N6, N7) 


pre RM[mod((rank+size) -1, size)].ready = true 
A RM[mod((rank+size) -1, size)].status = idle 
eff RM[mod((rank+size) -1, size)].status := receive 


output SEND(m, 12, 13) 
pre P.status # idle A m € (P.pending) 
eff SM[mod(rank + 1, size)].toSend := 
(SM[mod(rank + 1, size)].toSend) | m; 


P.pending := delete(m, P.pending) 
input resp_Iprobe(flag, N4, N5) 
eff RM[mod((rankt+tsize) -1, size)].ready := flag; 
RM[mod((rank+size) -1, size)].status := idle 


input resp_test(flag, N18, N19) 
eff if flag = true then 
SM[mod(rank + 1, size)].handles := 
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tail (SM[mod(rank + 1, size)].handles); 
SM[mod(rank + 1, size)].sent := 
tail (SM[mod(rank + 1, size)].sent) 
fi; 
SM[mod(rank + 1, size)].status := idle 
input resp_receive(m, N8, NQ) 
eff RM[mod((rank+size) -1, size)].toRecv := 
(RM[mod((rank+size) -1, size)].toRecv) | m; 


RM[mod((rankt+size) -1, size)].ready := false; 
RM[mod((rank+size) -1, size)].status := idle 
input vote 
eff P.status := voting 


output test(handle, N16, N17) 
pre SM[mod(rank + 1, size)].status = idle 


A handle = head(SM[mod(rank + 1, size)].handles) 


eff SM[mod(rank + 1, size)].status := test 
output Iprobe(N2, N3) 
pre RM[mod((rank+size) -1, size)].status = idle 
A RM[mod((rank+size) -1, size)].ready = false 
eff RM[mod((rank+size) -1, size)].status := Iprobe 
output RECEIVE(m, I0, I1) where m > I1 Vm<I1Vm=Tt 
pre m = head(RM[mod((rank+size) -1, size)].toRecv) 
eff if m > 11 then P.pending := insert(m, P. pending) 
elseif m = I1 then P.status := elected 
fi; 


RM[mod((rank+size) -1, size)].toRecv := 
tail (RM[mod((rank+size) -1, size)].toRecv) 
output leader (I4) 
pre P.status = elected 
eff P.status := announced 
input resp_Isend(handle, N14, N15) 
eff SM[mod(rank + 1, size)].handles := 
(SM[mod(rank + 1, size)].handles) | handle; 
SM[mod(rank + 1, size)].status := idle 
output Isend(m, N12, N13) 
pre head(SM[mod(rank + 1, size)].toSend) =m 
A SM[mod(rank + 1, size)].status = idle 
eff SM[mod(rank + 1, size)].toSend := 
tail (SM[mod(rank + 1, size)].toSend); 
SM[mod(rank + 1, size)].sent := 
(SM[mod(rank + 1, size)].sent) | m; 
SM[mod(rank + 1, size)].status := Isend 
schedule 
do 
fire input vote; 
while (true) do 
if P.pending # {} then 
fire output SEND (chooseRandom(P.pending), rank, 
if SM[Cmod(rank+1i,size)].status = idle 
A SM[mod(rank+1,size)].toSend # {} then 
fire output 
Isend (head (SM[mod(rank+1i,size)].toSend), rank, 
if SM[Cmod(rank+1i,size)].status = idle 
A SM[mod(rankt+1,size)].handles # {} then 
fire output 
test (head (SM[mod(rank+1,size)].handles), rank, 
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mod(rank+i,size)) fi; 


mod(rank+1i,size)) fi; 


mod(rank+1,size)) fi; 


if RM[mod((rank+size) -1,size)].status = idle 
A RM[mod((rank+size) -1,size)].ready = false then 
fire output Iprobe(rank, mod(rank-1,size)) fi; 
if RM[mod((rank+size) -1,size)].status = idle 
A RM[mod((rank+size) -1,size)].ready = true then 
fire output receive(rank, mod((rankt+size) -1, size)) fi; 
if RM[mod((rank+size) -1,size)].toRecv # {} then 
fire output 
RECEIVE (head (RM[mod((rank+size) -1,size)].toRecv), 
mod((rank+size) -1,size), rank) fi; 
if P.status = elected then 
fire output leader(rank) fi 


od 
od 
type Status = enumeration of idle, voting, elected, announced 
type rCall = enumeration of idle, receive, Iprobe 
type sCall = enumeration of idle, Isend, test 


type _States[LCRProcess] = tuple of pending: Set[Int], status: Status 

type _States[ReceiveMediator, Int, Int] = tuple of status: rCall, toRecv: 
Seq[Int], ready: Bool 

type _States[SendMediator, Int, Int] = tuple of status: sCall, toSend: 
Seq[Int], sent: Seq[Int], handles: Seq [Handle] 


C.3 Asynchronous Spanning Tree 


Expanded automaton with initialization and scheduling 


type Message = enumeration of search, null 
type rCall = enumeration of idle, receive, Iprobe 
type sCall = enumeration of idle, Isend, test 


type _States[sTreeProcess] = tuple of parent: Int, reported: 
Bool, send: Map[Int, Message] 


type _States[SendMediator, Message, Int] = tuple of status: sCall, toSend: 
Seq[Message], sent: Seq[Message], handles: Seq[Handle] 
type _States[ReceiveMediator, Message, Int] = tuple of status: rCall, toRecv: 


Seq[Message], ready: Bool 
uses ChoiceSet (Int) 


automaton sTree(rank: Int, nbrs: Set[Int]) 

signature 
output SEND(m: Message, v10: Int, vii: Int) where v10 = rank 
output RECEIVE(m: Message, vO: Int, vi: Int) where vO = rank 
output PARENT(j3: Int) 
output test(handle: Handle, vi6: Int, vi7: Int) where vi6 = rank 
input resp_test(flag: Bool, v1i8: Int, v19: Int) where v18 = rank 
output receive(v6: Int, v7: Int) where v6 = rank 
input resp_receive(m: Message, v8: Int, v9: Int) where v8 = rank 
output Iprobe(v2: Int, v3: Int) where v2 = rank 
input resp_Iprobe(flag: Bool, v4: Int, v5: Int) where v4 = rank 
output Isend(m: Message, vi2: Int, vi3: Int) where vi2 = rank 
input resp_Isend(handle: Handle, vi4: Int, vi5: Int) where vi4 = rank 

states 
P: _States[sTreeProcess], 
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RM: Map[Int, _States[ReceiveMediator, Message, Int]], 


SM: Map[Int, _States[SendMediator, Message, Int]], 
% Temporary 
tempNbrs: Set[Int], j: Int 
initially 
true > P.parent = -1 
det do 
P.parent := -1; 
P.reported := false; 
tempNbrs := nbrs; 
while (7isEmpty(tempNbrs)) do 
j := chooseRandom (tempNbrs) ; 
tempNbrs := delete(j, tempNbrs); 
if rank = 0 then 
P.send[j] := search 
else 
P.send[j] := null 
fi; 
RM[Cj]] := Lidle, {}, false]; 
SM[j] [idle, {}, {}, {}] 
od 
od 


transitions 
output SEND(m, vi0O, vii) where vi0 = rank 
pre P.send[vii] = search 
eff SM[vii].toSend := (SM[vii].toSend) | mn; 
P.send[vi1] := null 
output RECEIVE(m, v0, vi) 
pre m = head(RM[v1].toRecv) 
eff if rank # 0 A P.parent = -1 then 
P.parent := vil; 
for k: Int in nbrs - {vi} do 
P.send[k] := search 
od 
fi; 
RM[vi].toRecv := tail (RM[vi].toRecv) 
output PARENT(j3) 
pre P.parent = j3 A P.reported = false 
eff P.reported := true 
output receive(v6, v7) 
pre RM[v7].ready = true A RM[v7].status = idle 


eff RM[v7].status := receive 
input resp_receive(m, v6, v7) 
eff RM[v7].toRecv := (RM[v7].toRecv) | m; 
RM[v7].ready := false; 
RM[v7].status := idle 


output test(handle, v16, v1i7) 


pre SM[v17].status = idle A handle = head(SM[v17].handles) 


eff SM[v17].status := test 
input resp_test(flag, v18, v19) 
eff if flag = true then 


SM[v1i9].handles := tail(SM[vi9].handles); 
SM[v19].sent := tail (SM[v19].sent) 

fi; 

SM[v19].status := idle 
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output Iprobe(v2, v3) 
pre RM[v3].status = idle A RM[v3].ready = false 


eff RM[v3].status := Iprobe 
input resp_Iprobe(flag, v2, v3) 
eff RM[v3].ready := flag; 
RM[v3].status := idle 


output Isend(m, vi2, v13) 
pre head(SM[v13].toSend) =m A SM[v13].status = idle 


eff SM[v13].toSend := tail (SM[vi3].toSend) ; 
SM[vi3].sent := (SM[vi3].sent) | @m; 
SM[v1i3].status := Isend 
input resp_Isend(handle, v12, v13) 
eff SM[v13].handles := (SM[v1i3].handles) | handle; 
SM[v1i3].status := idle 
schedule 
states 
nb: Set[Int], 
k: Int 
do 
while (true) do 
nb := nbrs; 
while (7isEmpty(nb)) do 
k := chooseRandom (nb); 
nb := delete(k, nb); 
if P.send[k] = search then 
fire output SEND(search, rank, k) fi; 
if SM[k].status = idle A SM[k].toSend # {} then 
fire output Isend(head(SM[k].toSend), rank, k) fi; 
if SM[k].status = idle A SM[k].handles # {} then 
fire output test (head(SM[k].handles), rank, k) fi; 
if RM[k].status = idle A RM[k].ready = false then 
fire output Iprobe(rank, k) fi; 
if RM[k].status = idle A RM[k].ready = true then 
fire output receive(rank, k) fi; 
if RM[k].toRecv 4 {} then 
fire output RECEIVE (head(RM[k].toRecv), rank, k) fi; 
if P.parent = k A P.reported = false then 
fire output PARENT(k) fi 
od 
od 
od 


C.4 Asynchronous Broadcast Convergecast 


Expanded automaton with initialization and scheduling 


type Kind = enumeration of bcast, ack 

type BCastMsg = tuple of kind: Kind , w: Int 

type Message = union of msg: BCastMsg, kind: Kind 

type rCall = enumeration of idle, receive, Iprobe 

type sCall = enumeration of idle, Isend, test 

type _States[bcastProcess] = tuple of val:Int, acked: Set[Int], 
parent: Int, reported: 
Bool, send: Map[Int, Seq[Message]], temp: BCastMsg 

type _States[SendMediator, Message, Int] = tuple of status: sCall, 
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toSend: 


Seq[Message], sent: Seq[Message], handles: Seq[Handle] 


type _States[ReceiveMediator, Message, Int] = tuple of status: rCall, toRecv: 


Seq[Message], ready: Bool 
uses ChoiceSet (Int) 


automaton bcast(rank: Int, nbrs: Set[Int]) 
assumes bcastNode_Axioms 
assumes P_Axioms 
assumes RM_Axioms 
assumes SM_Axioms 
signature 
output receive(v6: Int, v7: Int) where v6 = rank 
output SEND(m: Message, v10: Int, vit: Int) where vi0 = rank 


input resp_Iprobe (flag: Bool, v4: Int, v5: Int) where v4 = rank 


internal report(v2: Int) where v2 = rank 


input resp_test(flag: Bool, vi8: Int, vi9: Int) where vi8 = rank 
input resp_receive(m: Message, v8: Int, v9: Int) where v8 = rank 
output test(handle: Handle, vi6: Int, vi7: Int) where vi6 = rank 


output Iprobe(v2: Int, v3: Int) where v2 = rank 

output RECEIVE(m: Message, vO: Int, vi: Int) where vO = rank 

input resp_Isend(handle: Handle, v1i4: Int, vi5: Int) where v14 

output Isend(m: Message, vi2: Int, vi3: Int) where vi2 = rank 
states 

P: _States[bcastProcess], 

RM: Map[Int, _States[ReceiveMediator, Message, Int]], 

SM: Map[Int, _States[SendMediator, Message, Int]], 

&4;Temporary variables 


tempNbrs : Set[Int] := f{}, 
j: Int 
initially 
true > P.val = 99 
det do 
if rank = 0 then 
P.val := 99; 
(P.temp).kind := bcast; 
(P.temp).w := P.val; 
tempNbrs := nbrs; 
while (7isEmpty(tempNbrs)) do 
j := chooseRandom (tempNbrs) ; 
tempNbrs := delete(j, tempNbrs) ; 
P.send[j] := {} F msg(P.temp) 
od 
else 
Pval ‘t=-—-15 
tempNbrs := nbrs; 
while (7isEmpty(tempNbrs)) do 
j := chooseRandom (tempNbrs) ; 
tempNbrs := delete(j, tempNbrs); 
P.send[j] := {} 
od 
fi; 
P.parent := -1; 
P.reported := false; 
P.acked := {}; 
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rank 


tempNbrs := nbrs; 

while (4isEmpty(tempNbrs)) do 
j := chooseRandom (tempNbrs) ; 
tempNbrs := delete(j, tempNbrs) ; 
RM[Cj]] := Lidle, {}, false]; 
SM[j] [idle, {}, {}, {}] 

od 


od 


transitions 
output receive(v6, v7) 
pre RM[v7].ready = true A RM[v7].status = idle 
eff RM[v7].status := receive 
output SEND(m, vi0, vii) where vi0O = rank 
pre m = head(P.send[vi1]) 
eff SM[vii].toSend := (SM[vii].toSend) | nm; 
P.send[vii] := tail(P.send[vi1i]) 
input resp_Iprobe(flag, v4, v5) 
eff RM[v5].ready := flag; 
RM[v5].status := idle 
internal report (v2) 
4; The preconditions were (incorrectly) removed 


eff if rank = 0 then P.reported := true 
elseif rank # 0 then 
P.send[P.parent] := P.send[P.parent] + kind(ack); 
P.reported := true 
fi 


input resp_test(flag, v18, v19) 
eff if flag = true then 


SM[v1i9].handles := tail(SM[vi9].handles); 
SM[v19].sent := tail (SM[v19].sent) 
fi; 
SM[v19].status := idle 
input resp_receive(m, v8, v9) 
eff RM[v9].toRecv := (RM[v9].toRecv) | m; 
RM[v9].ready := false; 
RM[v9].status := idle 


output test(handle, v16, v1i7) 


pre SM[v1i7].status = idle A handle = head(SM[v17].handles) 


eff SM[v17].status := test 
output Iprobe(v2, v3) 
pre RM[v3].status = idle A RM[v3].ready = false 
eff RM[v3].status := Iprobe 
output RECEIVE(m, v0, vi) 
pre m = head(RM[v1].toRecv) 


eff if m = kind(ack) then P.acked := (P.acked) U {vi} 
else 
if P.val = -1 then 
P.val := (m.msg).w; 
P.parent := vil; 
for k: Int in (nbrs) - {vi} do 
P.send[k] := P.send[k] | m 
od 
else P.send[vi] := P.send[vi] + kind (ack) 
fi 
fi; 
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RM[vi].toRecv := tail (RM[v1].toRecv) 
input resp_Isend(handle, v14, v15) 
eff SM[v15].handles := (SM[vi5].handles) | handle; 
SM[vi5].status := idle 
output Isend(m, v12, v13) 
pre head(SM[v13].toSend) =m A SM[v13].status = idle 
eff SM[v13].toSend := tail (SM[vi3].toSend) ; 
SM[vi3].sent := (SM[v1i3].sent) - @m; 
SM[v13].status := Isend 


schedule 
states 
tempNbrs2: Set[Int], 
k: Int 
do 
while(true) do 
tempNbrs2 := nbrs; 
while (7isEmpty(tempNbrs2)) do 
k := chooseRandom (tempNbrs2); 
tempNbrs2 := delete(k, tempNbrs2); 
if P.send[k] #~ {} then 
fire output SEND(head(P.send[k]), rank, k) fi; 
if SM[k].status = idle A SM[k].toSend # {} then 
fire output Isend(head(SM[k].toSend), rank, k) fi; 
if SM[k].status = idle A SM[k].handles # {} then 
fire output test (head(SM[k].handles), rank, k) fi; 
if RM[k].status = idle A RM[k].ready = false then 
fire output Iprobe(rank, k) fi; 
if RM[k].status = idle A RM[k].ready = true then 
fire output receive(rank, k) fi; 
if RM[k].toRecv # {} then 
fire output RECEIVE (head(RM[k].toRecv), rank, k) fi 
od; 
if rank = 0 A P.acked = nbrs A P.reported = false then 
fire internal report(rank) fi; 
if rank #£ 0 A P.parent # -1 A P.acked = nbrs - {P.parent} A 
P.reported = false then 
fire internal report(rank) fi 
od 
od 


C.5 Leader Election using Asynchronous Broadcast Convergecast 


Expanded automaton with initialization and scheduling 


type Kind = enumeration of bcast, ack 

type BCastMsg = tuple of kind: Kind, w: Int 

type AckMsg = tuple of kind: Kind, mx: Int 

type MSG = union of bmsg: BCastMsg, amsg: AckMsg, kind: Kind 
type Message = tuple of msg: MSG, source: Int 


type rCall = enumeration of idle, receive, Iprobe 
type sCall = enumeration of idle, Isend, test 


type _States[bcastLeaderProcess] = tuple of val: Map[Int, Int], 
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parent: Map[Int, Int], reported: Map[Int, Bool], acked: 
Map[Int, Set[Int]], send: Map[Int, Int, Seq[Message]], max: 
Map[Int, Int], elected: Bool, announced: Bool 
type _States[ReceiveMediator, Message, Int] = tuple of status: rCall, toRecv: 
Seq[Message], ready: Bool 
type _States[SendMediator, Message, Int] = tuple of status: sCall, toSend: 


Seq[Message], sent: 


type _States[bcastLeaderNod 
RM: Map[Int, _States[Rece 
Map[Int, _States[SendMedi 


uses ChoiceSet (Int) 


automaton bcastLeader(rank: Int, size:Int, nbrs:Set[Int]) 
signature 
output receive(N6: Int, N7: Int) where N6 = rank 
output SEND(m: Message, N10: Int, N11: Int) where N10 = rank 
input resp_Iprobe(flag: Bool, N4: Int, N5: Int) where N4 = rank 
internal finished 
internal report(I2: Int, source: Int) where I2 = rank 
output LEADER 
input resp_test(flag: Bool, N18: Int, N19: Int) where N18 = rank 
input resp_receive(m: Message, N8: Int, N9: Int) where N8 = rank 
output test(handle: Handle, N16: Int, N17: Int) where Ni6 = rank 
output Iprobe(N2: Int, N3: Int) where N2 = rank 
output RECEIVE(m: Message, NO: Int, Ni: Int) where NO = rank 
input resp_Isend(handle: Handle, N14: Int, N15: Int) where N14 = 
output Isend(m: Message, N12: Int, N13: Int) where N12 = rank 
states 
P: _States[bcastLeaderProcess], 
RM: Map[Int, _States[ReceiveMediator, Message, Int]], 
SM: Map[Int, _States[SendMediator, Message, Int]], 
tempNbrs: Set[Int], 
ce Int, 
j: Int 
initially 
(true 
=> 
Voj: Int 
(O0< j Aj < 16 
=> 
(rank = j 
=> 
P.val[j] = rank 
A (rank #A j => P.val[j] = -1) 
A P.parent[j] = -1 
A P.acked[j] = {} 
A P.max[j] = rank 
\ 
Vk: Int 
(P.send[j, k] = {} 
A k € nbrs 


Seq[Message], 


handles: Seq[Handle] 
e] = tuple of P: 
iveMediator, Message, 
ator, Message, Int]] 


Int]], SM: 


Al 


_States [bcastLeaderProcess], 


rank 


A rank = j 
=> 
P.send[j, k] 
= {} -F [bmsg([bcast, 99]), j]))) 
A P.elected = false) 
A 
Voj: Int 
(RM[j].status = idle 
A RM[j].toRecv = {} 
A RM[j]l.ready = false 
A defined(RM, j)) 
x 
Voj: Int 
(SM[j].status = idle 
A SM[j].toSend = {} 
A SM[j].sent = {} 
A SM[j].handles = {} 
A defined(SM, j)) 
det do 
c := size; 
while (c > 0) do 
Cc 1S oe 43 
if (rank = c) then 
P.val[c] := rank 
else 
P.val[c] := -1 
fi; 
.parent[c] := -1; 
.acked[c] := {}; 
.max[c] := rank; 
.reported[c] := false; 
-elected := false; 
.-announced := false; 


uv UU UU 


tempNbrs := nbrs; 
while (7isEmpty(tempNbrs)) do 
j := chooseRandom (tempNbrs) ; 
tempNbrs := delete(j, tempNbrs); 
P.send[c, j] := {}; 
if c = rank then 
P.send[c, j] := {} F [bmsg( [bcast, 99] ), c] 
fi 
od; 


tempNbrs := nbrs; 

while (7isEmpty(tempNbrs)) do 
j := chooseRandom (tempNbrs) ; 
tempNbrs := delete(j, tempNbrs); 
RM[j] := [idle, {}, false]; 
SM[j] := L[idle, {}, {}, {}] 

od 

od 
od 


transitions 
output receive(N6, N7) 
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pre RM[N7].ready = true A RM[N7].status = idle 
eff RM[N7].status := receive 

output SEND(m, N10, N11) where N10 = rank 
pre m = head(P.send[m.source, N1i1]) 


eff SM[N11].toSend := (SM[N1ii].toSend) | mn; 
P.send[m.source, Nii] := tail(P.send[m.source, N1i1i]) 
input resp_Iprobe(flag, N4, N5) 
eff RM[N5].ready := flag; 
RM[N5].status := idle 


output LEADER 
pre P.elected = true A P.announced = false 
eff P.announced := true 


internal finished 
% preconditions didn’t pass through 
pre P.acked[rank] = nbrs A 
P.reported[rank] = false 


eff P.reported[rank] := true; 
if (P.max[rank] = rank) then 
P.elected := true 
fi; 


internal report(I2, source) 
eff P.send[source, P.parent[source]] := 
P.send[source, P.parent[source]] 
- [Lamsg([Lack, P.max[source]]), source]; 
P.reported[source] := true 


input resp_test(flag, N18, N19) 
eff if flag = true then 


SM[N19].handles := tail(SM[N19].handles); 
SM[N19].sent := tail (SM[N19].sent) 
fi; 
SM[N19].status := idle 
input resp_receive(m, N8, NQ) 
eff RM[N9].toRecv := (RM[N9].toRecv) | m; 
RM[N9].ready := false; 
RM[N9].status := idle 


output test(handle, N16, N17) 
pre SM[N17].status = idle A handle = head(SM[N17].handles) 
eff SM(N17].status := test 
output Iprobe(N2, N3) 
pre RM[N3].status = idle A RM[N3].ready = false 
eff RM[N3].status := Iprobe 
output RECEIVE(m, NO, N1) 
pre m = head(RM[N1].toRecv) 
eff if m.msg = kind(ack) then 
P.acked[m.source] := P.acked[m.source] U {N1} 
elseif tag(m.msg) = amsg then 
if P.max[m.source] < (((m.msg).amsg).mx) then 


P.max[m.source] := ((m.msg).amsg).mx 
fi; 
P.acked[m.source] := P.acked[m.source] U {Ni} 
else 
if P.val[m.source] = -1 then 
P.val[m.source] := ((m.msg).bmsg).w; 
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P.parent[m.source] := N1; 
for k: Int in (nbrs) - {Ni} do 
P.send[m.source, k] := P.send[m.source, k] | m 
od 
else 
P.send[m.source, Ni] := 
P.send[m.source, Ni] | [kind(ack), m.source] 


fi 
fi; 
RM[N1].toRecv := tail (RM[N1].toRecv) 
input resp_Isend(handle, N14, N15) 
eff SM[N15].handles := (SM[N1i5].handles) | handle; 
SM[N15].status := idle 


output Isend(m, N12, N13) 
pre head(SM[N13].toSend) = m A SM[N13].status = idle 


eff SM[N13].toSend := tail (SM[N13].toSend) ; 
SM[N13].sent := (SM[N1i3].sent) - @m; 
SM[N13].status := Isend 
schedule 
states 


c2: Int, %% Source 
tempNbrs2: Set[Int], 
k: Int 


do 
while(true) do 
c2 := size; 
while (c2 > 0) do 
62 t= 62 >= 14 
tempNbrs2 := nbrs; 
while (7isEmpty(tempNbrs2)) do 
k := chooseRandom(tempNbrs2) ; 
tempNbrs2 := delete(k, tempNbrs2); 
if P.send[c2, k] 4 {} then 
fire output SEND(head(P.send[c2, k]), rank, k) fi; 
if SM[k].status = idle A SM[k].toSend # {} then 
fire output Isend(head(SM[k].toSend), rank, k) fi; 
if SM[k].status = idle A SM[k].handles # {} then 
fire output test (head(SM[k].handles), rank, k) fi; 
if RM[k].status = idle A RM[k].ready = false then 
fire output Iprobe(rank, k) fi; 
if RM[k].status = idle A RM[k].ready = true then 
fire output receive(rank, k) fi; 
if RM[k].toRecv # {} then 
fire output RECEIVE(head(RM[k].toRecv), rank, k) fi 
od; 
if c2 #A rank A P.parent[c2] # -1 A 
P.acked[c2] = nbrs - {P.parent[c2]} A 
P.reported[c2] = false then 
fire internal report(rank, c2) fi; 
if c2 = rank A P.acked[rank] = nbrs A 
P.reported[rank] = false then 
fire internal finished fi; 
if P.elected = true A P.announced = false then 
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fire output LEADER fi 
od 
od 
od 


C.6 Spanning Tree to Leader Election 


Expanded automaton with initialization and scheduling 
uses ChoiceSet (Int) 


automaton sTreeLeader(rank: Int, nbrs: Set[Int]) 
signature 
output receive(N6: Int, N7: Int) where N6 = rank 
output SEND(m: Message, N10: Int, N11: Int) where N10 = rank 


input resp_Iprobe(flag: Bool, N4: Int, N5: Int) where N4 = rank 
input resp_test(flag: Bool, N18: Int, N19: Int) where N18 = rank 
input resp_receive(m: Message, N8: Int, N9: Int) where N8 = rank 
output test(handle: Handle, N16: Int, N17: Int) where Ni6 = rank 


output Iprobe(N2: Int, N3: Int) where N2 = rank 
output RECEIVE(m: Message, NO: Int, Ni: Int) where NO = rank 
output leader 


input resp_Isend(handle: Handle, N14: Int, N15: Int) where N14 


output Isend(m: Message, N12: Int, N13: Int) where N12 = rank 
states 
P: _States[sTreeLeaderProcess], 
RM: Map[Int, _States[ReceiveMediator, Message, Int]], 
SM: Map[Int, _States[SendMediator, Message, Int]], 
% Temporary storage 
j: Int, 
tempNbrs: Set[Int] 
initially 
(true => 
P.receivedElect = {} 
A P.sentElect = {} 
A P.status = idle 
A (size(nbrs) = 15> 
P.send[chooseRandom (nbrs) ] 
= P.send[chooseRandom(nbrs)] + elect)) 
AW j: Int 
(j € nbrs => 
(RM[j].status = idle 
A RM[j].toRecv = {} 
A RM[j]l.ready = false 
A defined(RM, j))) 
AW j: Int 
(j € nbrs => 
(SM[j].status = idle 
A SM[j].toSend = {} 
A SM[jl]l.sent = {} 
A SM[j].handles = {} 
A defined(SM, j))) 
det do 
P.receivedElect := {}; 
P.sentElect := {}; 
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rank 


P.status := idle; 
tempNbrs := nbrs; 


while 
j 


(jisEmpty(tempNbrs)) do 
chooseRandom (tempNbrs) ; 


tempNbrs := delete(j, tempNbrs); 
P.send[j] := {}; 

RM[j] := [idle, {}, false]; 
SM[j] := [idle, {}, {}, {}] 


od; 


if size(nbrs) = 1 then 
P.send[chooseRandom(nbrs)] := {} - elect 


fi 
od 


transitions 
output receive(N6, N7) 


pre 
eff 


RM[N7].ready = true A RM[N7].status = idle 
RM(N7].status := receive 


output SEND(m, N10, N11) where N10 = rank 


pre 
eff 


input 
eff 


input 
eff 


input 
eff 


m = head(P.send[Ni1]) 

SM[N1ii].toSend := (SM[N1i1].toSend) | @m; 
P.send[Ni1] := tail(P.send[N11]) 
resp_Iprobe(flag, N4, N5) 

RM[N5].ready := flag; 

RM[N5].status := idle 

resp_test(flag, N18, N19) 

if flag = true then 


SM[N19].handles := tail(SM[N1i9].handles); 
SM[N19].sent := tail (SM[N19].sent) 

fi; 

SM[N19].status := idle 

resp_receive(m, N8, N9) 

RM[N9].toRecv := (RM[N9].toRecv) | m; 

RM[N9].ready := false; 

RM[N9].status := idle 


output test(handle, N16, N17) 


pre 
eff 


SM[N1i7].status = idle A handle = head(SM[N17]. handles) 
SM[N17].status := test 


output Iprobe(N2, N3) 


pre 
eff 


RM[N3].status = idle A RM[N3].ready = false 
RM[(N3].status := Iprobe 


output RECEIVE(m, NO, N1) 


pre 
eff 


m = head(RM[N1].toRecv) 


P.receivedElect := insert(Ni, P.receivedElect ); 
if size(P.receivedElect) = size(nbrs) - 1 then 
P.send[chooseRandom(nbrs - (P.receivedElect))] := 
P.send[chooseRandom(nbrs - (P.receivedEFlect))] | elect; 
P.sentElect := 
insert 
(chooseRandom(nbrs - (P.receivedElect)), P.sentElect) 
elseif P.receivedElect = nbrs then 
if Ni € (P.sentElect) then 
if NO > Ni then P.status := elected fi 
else P.status := elected 
fi 
fi; 
RM[N1i].toRecv := tail (RM[N1].toRecv) 
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output leader 
pre P.status = elected 


eff P.status := announced 
input resp_Isend(handle, N14, N15) 
eff SM[N15].handles := (SM[N15].handles) | handle; 
SM[N15].status := idle 


output Isend(m, N12, N13) 
pre head(SM[N13].toSend) = m A SM[N13].status = idle 


eff SM[N13].toSend := tail (SM[N13].toSend) ; 
SM[N13].sent := (SM[N13].sent) - @m; 
SM[N13].status := Isend 
schedule 
states 
tempNbrs2: Set[Int], 
k: Int 
do 
while(true) do 
tempNbrs2 := nbrs; 
while (4isEmpty(tempNbrs2)) do 
k := chooseRandom (tempNbrs2); 
tempNbrs2 := delete(k, tempNbrs2); 
if P.send[k] #~ {} then 
fire output SEND(head(P.send[k]), rank, k) fi; 
if SM[k].status = idle A SM[k].toSend # {} then 
fire output Isend(head(SM[k].toSend), rank, k) fi; 
if SM[k].status = idle A SM[k].handles # {} then 
fire output test (head(SM[k].handles), rank, k) fi; 
if RM[k].status = idle A RM[k].ready = false then 
fire output Iprobe(rank, k) fi; 
if RM[k].status = idle A RM[k].ready = true then 
fire output receive(rank, k) fi; 
if RM[k].toRecv # {} then 
fire output RECEIVE (head(RM[k].toRecv), rank, k) fi 
od; 
if P.status = elected then fire output leader fi 
od 
od 
type Status = enumeration of idle, elected, announced 
type Message = enumeration of elect 
type rCall = enumeration of idle, receive, Iprobe 
type sCall = enumeration of idle, Isend, test 
type _States[sTreeLeaderProcess] = tuple of receivedElect: 


Set [Int], sentElect: Set[Int], status: Status, send: 
Map[Int, Seq[Message]] 

type _States[ReceiveMediator, Message, Int] = tuple of status: rCall, 
toRecv: Seq[Message], ready: Bool 

type _States[SendMediator, Message, Int] = tuple of status: sCall, toSend: 
Seq[Message], sent: Seq[Message], handles: Seq[Handle] 


C.7 GHS 


Expanded automaton with initialization and scheduling 


A7 


uses ChoiceSet (Link) 


automaton 


GHS(rank: Int, size: Int, links: Set[Link], weight: Map[Link, 


signature 
input resp_receive(m: Message, N8: Int, N9: Int) where N8 
internal ReceiveChangeRoot (qp: Link) 
input resp_Iprobe(flag: Bool, N4: Int, N5: Int) where N4 
output RECEIVE(m: Message, i: Int, j: Int) where i = rank 


Int]) 


rank 


rank 


input resp_Isend(handle: Handle, N14: Int, N15: Int) where N14 = rank 
rank 


output Isend(m: Message, N12: Int, N13: Int) where N12 = 
internal ReceiveConnect (qp: Link, 1: Int) 

internal ReceiveReport (qp: Link, w: Int) 

internal ReceiveAccept (qp: Link) 

internal ReceiveInitiate (qp: Link, 1: Int, c: Null[Edge], 
output NotInTree(1l: Link) 

output InTree(1: Link) 

output Iprobe(N2: Int, N3: Int) where N2 = rank 

internal ReceiveTest (qp: Link, 1: Int, c: Null [Edge]) 
input resp_test(flag: Bool, N18: Int, N19: Int) where N18 


output SEND(m: Message, N10: Int, Nii: Int) where N10 = rank 


output receive(N6: Int, N7: Int) where N6 = rank 
output test(handle: Handle, N16: Int, N17: Int) where N16 
internal ReceiveReject (qp: Link) 
input startP 
states 
P: _States[GHSProcess], 
RM: Map[Int, _States[ReceiveMediator, Message, Int]], 
SM: Map[Int, _States[SendMediator, Message, Int]], 
% Temporary 
tempLinks: Set[Link], tempL: Link 


initially 
(true 
=> 
P.nstatus = sleeping 
A P.nfrag = nil 
A P.nlevel = 0 
A P.bestlink = embed(chooseRandom (links) ) 
A P.bestwt = weight [chooseRandom (links) ] 
A P.testlink = nil 
A P.inbranch = chooseRandom (links) 
A P.findcount = 0 
\ 
Voi: Link 
(1 € links 
= 
P.1lstatus[1] = unknown 
A P.answered[1] = false 
A P.queueOut [1] = {} 
A P.queueIn[1] = {})) 
\ 
Voj: Int 


(RM[j].status = idle 
A RM[j].toRecv = {} 
A RM[j].ready = false 
A defined(RM, j)) 
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st: 


Status) 


rank 


rank 


/\ 
Voj: Int 
(SM[j].status = idle 
A SM[j].toSend = {} 
A SM[j].sent = {} 
A SM[j].handles = {} 
A defined(SM, j)) 


det do 

P.nstatus := sleeping; 

P.nfrag := nil; 

P.nlevel := 0; 

P.bestlink := embed(chooseRandom (links)); 

P.bestwt := weight [chooseRandom (links)]; 

P.testlink := nil; 

P.inbranch := chooseRandom (links) ; 

P.findcount: 2— 0; 

tempLinks := links; 

while (7isEmpty(tempLinks)) do 
tempL := chooseRandom(tempLinks) ; 
tempLinks := delete(tempL, tempLinks); 
P.1lstatus[tempL] := unknown; 
P.answered[tempL] := false; 
P.queueOut[tempL] := {}; 
P.queueIn[[tempL.t, tempL.s]] := {}; 
RM[tempL.t] := [idle, {}, false]; 
SM[tempL.t] := [idle, {}, {}, {}] 

od 

od 


transitions 
input resp_receive(m, N8, NQ) 


eff RM[N9].toRecv := (RM[N9].toRecv) | m; 
RM[N9].ready := false; 
RM[N9].status := idle 
internal ReceiveChangeRoot (qp) 
eff P.queueIn[qp] := tail(P.queueIn[qp]); 


if P.1lstatus[(P.bestlink).val] = branch then 
P.queueOut [(P.bestlink).val] := 
P.queueOut [(P.bestlink).val] - msg (CHANGEROOT ) 
else 
P.queueOut [(P.bestlink).val] := 
P.queueOut [(P. bestlink).val] 
F connMsg([CONNECT, P.nlevel]); 
P.1lstatus[(P.bestlink).val] := branch 
fi 
input resp_Iprobe(flag, N4, N5) 
eff RM[N5].ready := flag; 
RM[N5].status := idle 
output RECEIVE(m, i, j) 
pre m = head(RM[j].toRecv) 
eff P.queuveIn[[j, i]] := P.queveIn[[j, i]] -F m; 
RM[j].toRecv := tail (RM[j].toRecv) 
input resp_Isend(handle, N14, N15) 
eff SM[N15].handles := (SM[N15].handles) | handle; 
SM[N15].status := idle 
output Isend(m, N12, N13) 
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pre head(SM[N13].toSend) = m A SM[N13].status = idle 


eff SM[N13].toSend := tail (SM[N13].toSend) ; 
SM[N13].sent := (SM[N13].sent) - @m; 
SM[N13].status := Isend 


internal ReceiveConnect (qp, 1) 
pre head(P.queueIn[qp]) = connMsg([CONNECT, 1]) 


eff P.queueIn[qp] := tail(P.queueIn[qp]); 
if P.nstatus = sleeping then 
P.minL := embed(chooseRandom (links)); 
P.min := weight[(P.minL).val]; 


for tempL: Link in links do 
if weight[tempL] < (P.min) then 


P.minL := embed(tempL); 
P.min := weight [tempL] 
fi 
od; 
P.lstatus[(P.minL).val] := branch; 
P.nstatus := found; 


P.queueOut [(P.minL).val] := 
P.queueOut [(P.minL).val] - connMsg([CONNECT, 0]) 


fi; 
if 1 < (P.nlevel) then 
P.lstatus[[qp.t, qp.s]] := branch; 
if P.testlink ~ nil then 
P.queueOut[[qp.t, qp.s]] := 
P.queueOut [[gp.t, qp.s]] 
fF initMsg([INITIATE, P.nlevel, P.nfrag, find]); 
P.findcount := (P.findcount) + 1 
else 
P.queueOut[[gp.t, qp.s]] := 
P.queueOut [[gp.t, qp.s]] 
F initMsg([INITIATE, P.nlevel, P.nfrag, found]) 
fi 
else 
if P.1lstatus[[qp.t, gqp.s]] = unknown then 
P.queueIn[qp] := P.queueIn[qp] + connMsg([CONNECT, 1]) 
else 
P.queueOut [[gp.t, qp.s]] := 
P.queueOut [[gp.t, qp.s]] 
fk 
initMsg 
(CINITIATE, 
(P.nlevel) + 1, 
embed([qp.t, qp.s]), 
find]) 
fi 
fi 


internal ReceiveReport (qp, w) 
pre head(P.queueIn[qp]) = reportMsg([REPORT, w]) 


eff P.queueIn[qp] := tail(P.queueIn[qp]); 
if [qp.t, qp.s] # P.inbranch then 
P.findcount := (P.findcount) - 1; 
if w < (P.bestwt) then 
P.bestwt := w; 
P.bestlink := embed([qp.t, qp.s]) 
fi; 
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if P.findcount = 0 A P.testlink = nil then 
P.nstatus := found; 
P.queueOut [P.inbranch] := 
P.queueOut [P.inbranch] - reportMsg([REPORT, P.bestwt]) 
fi 
else 
if P.nstatus = find then 
P.queueIn[qp] := P.queueIn[qp] + reportMsg([REPORT, w]) 
elseif w > (P.bestwt) then 
if P.1lstatus[(P.bestlink).val] = branch then 
P.queueOut [(P.bestlink).val] := 
P.queueOut [(P.bestlink).val] + msg (CHANGEROOT ) 
else 
P.queueOut [(P.bestlink).val] := 
P.queueOut [(P. bestlink).val] 
F connMsg([CONNECT, P.nlevel]); 
P.1lstatus[(P.bestlink).val] := branch 
fi 
fi 
fi 
internal ReceiveAccept (qp) 
pre head(P.queueIn[qp]) = msg (ACCEPT) 


eff P.queueIn[qp] := tail(P.queueIn[qp]); 

P.testlink := nil; 

if weight[[qp.t, qp.s]] < (P.bestwt) then 
P.bestlink := embed([qp.t, qp.s]); 
P.bestwt := weight[[qp.t, qp.s]] 

fi; 

if P.findcount = 0 A P.testlink = nil then 
P.nstatus := found; 


P.queueOut [P.inbranch] := 
P.queueOut [P.inbranch] + reportMsg([REPORT, P.bestwt]) 
fi 
internal ReceiveInitiate(qp, 1, c, st) 
pre head(P.queueIn[qp]) = initMsg([INITIATE, 1, c, st]) 


eff P.queueIn[qp] := tail(P.queueIn[qp]); 
P.nlevel := 1; 
P.nfrag := c; 
if st = find then P.nstatus := find else P.nstatus := found fi; 
PivS = 035 


for pr: Link in links do 
if pr.t # qp.s A P.1lstatus[pr] = branch then 
P.S := (P.S) U {pr} 
fi 
od; 
for k: Link in P.S do 
P.queueOut [k] := P.queueOut[k] + initMsg([INITIATE, 1, c, st]) 
od; 
if st = find then 
P.inbranch := [qp.t, qp.s]; 
.bestlink := nil; 
.bestwt := 1000; 
minh += nals 
,min: <= 10007 
for tempL: Link in links do 
if weight[tempL] < (P.min) A P.1lstatus[tempL] = unknown 
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then 
P.minL := embed(tempL); 
P.min := weight [tempL] 
fi 
od; 
if P.minL # nil then 
P.testlink := P.minL; 
P.qgueueOut [(P.minL).val] := 
P.queueOut [(P.minL).val] 
F testMsg([TEST, P.nlevel, P.nfrag]) 


else 
P.testlink := nil; 
if P.findcount = 0 A P.testlink = nil then 
P.nstatus := found; 
P.queueOut [P.inbranch] := 
P.queueOut [P.inbranch] 
Ff reportMsg([REPORT, P.bestwt]) 
fi 
fi; 
P.findcount := size(P.S) 


fi 
output NotInTree(1) 
pre P.answered[1] = false A P.1lstatus[1] = rejected 
eff P.answered[1] := true 
output InTree(1) 
pre P.answered[1] = false A P.1status[1l] = branch 
eff P.answered[1] := true 
output Iprobe(N2, N3) 
pre RM[N3].status = idle A RM[N3].ready = false 
eff RM[N3].status := Iprobe 
internal ReceiveTest(qp, 1, c) 
pre head(P.queueIn[qp]) = testMsg([TEST, 1, c]) 


eff P.queueIn[qp] := tail(P.queueIn[qp]); 
if P.nstatus = sleeping then 
P.minL := embed(chooseRandom (links)); 
P.min := weight[(P.minL).val]; 


for tempL: Link in links do 
if weight[tempL] < (P.min) then 


P.minL := embed(tempL); 
P.min := weight [tempL] 
fi 
od; 
P.lstatus[(P.minL).val] := branch; 
P.nstatus := found; 


P.queueOut [(P.minL).val] := 
P.queueOut [(P.minL).val] + connMsg([CONNECT, 0]) 


fi; 
if 1 > (P.nlevel) then 

P.queueIn[qp] := P.queuveIn[qp] - testMsg([TEST, 1, c]) 
else 


if c # P.nfrag then 
P.qgueueOut[[qp.t, qp.s]] := 
P.queueOut [[qp.t, qp.s]] - msg(ACCEPT) 
else 
if P.1lstatus[[qp.t, qp.s]] = unknown then 
P.lstatus[[qp.t, qp.s]] := rejected 
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fi; 
if P.testlink # embed([qp.t, qp.s]) then 
P.queueOut[[gp.t, qp.s]] := 
P.queueOut[[qp.t, qp.s]] + msg (REJECT) 


else 
P.minL := nil; 
P.min := 1000; 


for tempL: Link in links do 
if weight[tempL] < (P.min) 
A P.1lstatus[tempL] = unknown then 
P.minL := embed(tempL); 
P.min := weight [tempL] 
fi 
od; 
if P.minL ~ nil then 
P.testlink := P.minL; 
P.queueOut [(P.minL).val] := 
P.queueOut [(P.minL). val] 
F testMsg([TEST, P.nlevel, P.nfrag]) 


else 
P.testlink := nil; 
if P.findcount = 0 A P.testlink = nil then 
P.nstatus := found; 
P.queueOut [P.inbranch] := 
P.queueOut [P.inbranch] 
F reportMsg([REPORT, P.bestwt]) 
fi 
fi 


fi 
fi 
fi 
input resp_test(flag, N18, N19) 
eff if flag = true then 


SM[N19].handles := tail(SM[N19].handles); 
SM[N19].sent := tail (SM[N19].sent) 

fi; 

SM[N19].status := idle 


output SEND(m, N10, N11) where N10 = rank 
pre m = head(P.queueOut[[N10, N1i1]]) 
eff SM[N11].toSend := (SM[N1ii].toSend) | nm; 
P.queueOut[[N10, N11]] := tail(P.queueOut[[N10, N11]]) 
output receive(N6, N7) 
pre RM[N7].ready = true A RM[N7].status = idle 
eff RM[N7].status := receive 
output test(handle, N16, N17) 
pre SM[N17].status = idle A handle = head(SM[N17].handles) 
eff SM[N17].status := test 
internal ReceiveReject (qp) 
pre head(P.queueIn[qp]) = msg (REJECT) 


eff P.queueIn[qp] := tail(P.queueIn[qp]); 
if P.1lstatus[[qp.t, gqp.s]] = unknown then 
P.lstatus[[qp.t, qp.s]] := rejected 
fi; 
Poni 2: na 4 
P. min += 1000; 


for tempL: Link in links do 
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if weight[tempL] < (P.min) A P.1lstatus[tempL] = unknown then 
P.minL := embed(tempL); 
P.min := weight [tempL] 
fi 
od; 
if P.minL # nil then 
P.testlink := P.minL; 
P.queueOut [(P.minL).val] := 
P.queueOut [(P.minL).val] 
F testMsg([TEST, P.nlevel, P.nfrag]) 


else 
P.testlink := nil; 
if P.findcount = 0 A P.testlink = nil then 
P.nstatus := found; 
P.queueOut [P.inbranch] := 
P.queueOut [P.inbranch] + reportMsg([REPORT, P.bestwt]) 
fi 
fi 


input startP 
eff if P.nstatus = sleeping then 
P.minL := embed(chooseRandom (links)); 
P.min := weight[(P.minL).val]; 
for tempL: Link in links do 
if weight[tempL] < (P.min) then 


P.minL := embed(tempL); 
P.min := weight [tempL] 
fi 
od; 
P.lstatus[(P.minL).val] := branch; 
P.nstatus := found; 


P.queueOut [(P.minL).val] := 
P.queueOut [(P.minL).val] - connMsg([CONNECT, 0]) 
fi 


schedule 
states 
inks: Set [Link], 
ink : Link 
do 
fire input startP; 
while(true) do 


Inks := links; 

while (7isEmpty(l1nks)) do 
lnk := chooseRandom (l1nks) ; 
links := delete(lnk, lnks); 


if P.queueOut [lnk] 4 {} then 
fire output SEND (head(P.queueOut [1nk]), rank, 1nk.t) fi; 
if SM[1nk.t].status = idle A SM[1nk.t].toSend # {} then 
fire output Isend(head(SM[1nk.t].toSend), rank, lnk.t) fi; 
if SM[1lnk.t].status = idle A SM[1nk.t].handles # {} then 
fire output test (head(SM[1nk.t].handles), rank, lnk.t) fi; 
if RM[1nk.t].status = idle A RM[1nk.t].ready = false then 
fire output Iprobe(rank, lnk.t) fi; 
if RM[1nk.t].status = idle A RM[1nk.t].ready = true then 
fire output receive(rank, lnk.t) fi; 
if RM[1Ink.t].toRecv # {} then 


54 


od 
od 


type 
type 
type 
type 
type 

CH 
type 
type 
type 
type 
type 
type 


fire output RECEIVE (head(RM[1nk.t].toRecv), rank, 1lnk.t) fi; 


if P.queueIn[[1nk.t, lnk.s]] # {} A 
tag (head (P.queueIn[[1nk.t, lnk.s]])) = connMsg then 
fire internal ReceiveConnect ([lnk.t, lnk.s], 
(head (P.queueIn[[1nk.t, Ink.s]])).connMsg.1) 
fi; 
if P.queueIn[[1lnk.t, lnk.s]] # {} A 
tag (head (P.queueIn[[1nk.t, lnk.s]])) = initMsg then 
fire internal ReceivelInitiate ([lnk.t, lnk.s], 
(head (P.queueIn[[1nk.t, lnk.s]])).initMsg.1, 
(head(P.queuveIn[[1nk.t, lnk.s]])).initMsg.c, 
(head(P.queuveIn[[1nk.t, Ilnk.s]])).initMsg.st) 
fi; 
if P.queueIn[[1nk.t, lnk.s]] # {} A 
tag(head(P.queueIn[[1nk.t, lnk.s]])) = testMsg then 
fire internal ReceiveTest ([1lnk.t, lnk.s], 
(head (P.queueIn[[1nk.t, 1lnk.s]])).testMsg.1, 
(head(P.queueIn[[1nk.t, lnk.s]])).testMsg.c) 
fi; 
if P.queueIn[[1lnk.t, lnk.s]] # {} A 
head (P.queueIn[[1nk.t, 1lnk.s]]) = msg(ACCEPT) then 
fire internal ReceiveAccept ([1lnk.t, lnk.s]) 
fi; 
if P.queueIn[[1lnk.t, lnk.s]] # {} A 
head (P.queueIn[[1nk.t, 1lnk.s]]) = msg(REJECT) then 
fire internal ReceiveReject ([lnk.t, 1lnk.s]) 
fi; 
if P.queueIn[[1nk.t, lnk.s]] # {} A 
tag (head (P.queueIn[[1nk.t, lnk.s]])) = reportMsg then 
fire internal ReceiveReport ([1nk.t, lnk.s], 
(head(P.queueIn[[1nk.t, 1lnk.s]])).reportMsg.w) 
fi; 
if P.queueIn[[1nk.t, lnk.s]] # {} A 
head (P.queueIn[[1nk.t, 1lnk.s]]) = msg(CHANGEROOT) then 
fire internal ReceiveChangeRoot ([1lnk.t, lnk.s]) 
fi; 
if P.answered[1lnk] = false A P.1status[1lnk] = branch then 
fire output InTree(l1nk) fi; 
if P.answered[1nk] = false A P.1lstatus[1nk] = rejected then 
fire output NotInTree(lnk) fi 


od 
Nstatus = enumeration of sleeping, find, found 
Edge = tuple of s: Int, t: Int 


Link = tuple of s: Int, t: Int 

Lstatus = enumeration of unknown, branch, rejected 

Msg = enumeration of CONNECT, INITIATE, TEST, REPORT, ACCEPT, 
ANGEROOT 

ConnMsg = tuple of msg: Msg, 1: Int 

Status = enumeration of find, found 

InitMsg = tuple of msg: Msg, 1: Int, c: Null[Edge], st: Status 
TestMsg = tuple of msg: Msg, 1: Int, c: Null [Edge] 

ReportMsg = tuple of msg: Msg, w: Int 


? 


REJECT, 


Message = union of connMsg: ConnMsg, initMsg: InitMsg, testMsg: 
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Msg 


TestMsg, reportMsg: ReportMsg, msg: 
type rCall = enumeration of idle, receive, Iprobe 
type sCall = enumeration of idle, Isend, test 
type _States[GHSProcess] = tuple of nstatus: Nstatus, nfrag: Null[Edge], 
nlevel: Int, bestlink: Null[Link], bestwt: Int, testlink: Null [Link], 
inbranch: Link, findcount: Int, lstatus: Map[Link, Lstatus], queueOut: 
Map[Link, Seq[Message]], queueIn: Map[Link, Seq[Message]], answered: 
Map[Link, Bool], min: Int, minL: Null[Link], S: Set[Link] 
type _States[ReceiveMediator, Message, Int] = tuple of status: rCall, 
toRecv: Seq[Message], ready: Bool 
type _States[SendMediator, Message, Int] = tuple of status: sCall, toSend: 
sent: Seq[Message], handles: Seq[Handle] 


Seq [Message], 


D RuntimeTables 


Table D.1: Runtime results of the LCR Leader Election Algorithm 
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Table D.2: Runtime results of the Asynchronous Spanning Tree algorithm 


Table D.3: Runtime results of the Asynchronous Broadcast Convergecast algorithm 
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Table D.4: Runtime results of the Leader Election using the Asynchronous Broadcast Convergecast 
Algorithm 


Table D.6: Runtime results of the GHS Minimum Spanning Tree algorithm 
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E Traces of runs 


E.1 LCR on 8 nodes 


Complete trace 


Initialization starts (0) on loon.csail.mit.edu at 7:25:29:615 
Modified state variables: 
P — [pending: (), status: idle] 
RM — Map{} 
SM — Map{} 
rank — null 
size — null 
Initialization ends 
transition: input vote() in automaton LCR(0) 
on loon.csail.mit.edu at 7:25:29:625 
Modified state variables: 
P — [pending: (0), status: voting] 
RM — Map{[7 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[1 -> [status: idle, toSend: {}, sent: {}, handles: {}]] } 
rank — 0 
size — 8 


transition: output SEND(0O, 0, 1) in automaton LCR(0) 
on loon.csail.mit.edu at 7:25:29:629 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[1 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {0 } Elements removed: {}] }]} 


Initialization starts (2) on parrot.csail.mit.edu at 7:25:29:820 
Modified state variables: 
P — [pending: (), status: idle] 
RM — Map{} 
SM — Map{} 
rank — null 
size — null 
Initialization ends 
Initialization starts (3) on tui.csail.mit.edu at 7:25:29:927 
transition: input vote() in automaton LCR (2) 
on parrot.csail.mit.edu at 7:25:29:960 
Modified state variables: 
P — [pending: (2), status: voting] 
RM — Map{[1 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[3 -> [status: idle, toSend: {}, sent: {}, handles: {}]] } 
rank — 2 
size — 8 


Modified state variables: 

P — [pending: (), status: idle] 

RM — Map{} 

SM — Map{} 

rank — null 

size — null 

Initialization ends 

Initialization starts (7) on tui.csail.mit.edu at 7:25:30:169 


59 


transition: output SEND(2, 2, 3) in automaton LCR(2) 


on parrot.csail.mit.edu at 7:25:30:200 


Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[3 -> Tuple, 


Sequence, 


elements added: {2 } Elements removed: 


Modified state variables: 


P — 


[pending: (), status: idle] 


RM — Map{} 
SM — Map{} 
rank — null 


size — null 


Initialization ends 


modified fields: 
{}] }]} 


{[toSend 


Initialization starts (6) on parrot.csail.mit.edu at 7:25:30:247 
Modified state variables: 


P — 


[pending: (), status: idle] 


RM — Map{} 
SM — Map{} 
rank — null 


size — null 


Initialization ends 


Initialization starts 


transition: input vote() in automaton LCR(6) 


on parrot.csail.mit.edu at 7:25:30:313 


Modified state variables: 


P — 


[pending: (6), status: voting] 


RM — Map{[5 -> [status: idle, toRecv: {}, 
SM — Map{I[7 -> [status: idle, toSend: {}, 
rank — 6 


size — 8 


Modified state variables: 


P — 


[pending: (), status: idle] 


RM — Map{} 
SM — Map{} 
rank — null 


size — null 


Initialization ends 
transition: input vote() in automaton LCR(5) 


on condor.csail.mit.edu at 7:25:30:426 


Modified state variables: 


P — 


[pending: (5), status: voting] 


RM — Map{[4 -> [status: idle, toRecv: {}, 
SM — Map{[6 -> [status: idle, toSend: {}, 
rank — 5 


size 


— 8 


ready: 
sent: 


ready: 
sent: 


transition: input vote() in automaton LCR(3) 


on tui.csail.mit.edu at 7:25:30:460 


Modified state variables: 


P => 


[pending: (3), status: voting] 


RM — Map{[2 -> [status: idle, toRecv: {}, 
SM — Map{[4 -> [status: idle, toSend: {}, 
rank — 3 


size 


— 8 
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ready: 
sent: 


false]] } 


{}, 


handles: 


false]] } 


{}, 


handles: 


false]] } 


£5 


handles: 


(5) on condor.csail.mit.edu at 7:25:30:286 


{}]] } 
{}]] } 
{}]] } 


=2 


transition: output SEND(6, 6, 7) in automaton LCR(6) 
on parrot.csail.mit.edu at 7:25:30:497 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[7 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {6 } Elements removed: {}] }]} 


transition: input vote() in automaton LCR(7) 
on tui.csail.mit.edu at 7:25:30:710 
Modified state variables: 
P — [pending: (7), status: voting] 
RM — Map{[6 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[0 -> [status: idle, toSend: {}, sent: {}, handles: {}]] } 
rank — 7 
size — 8 


transition: output SEND(5, 5, 6) in automaton LCR(5) 
on condor.csail.mit.edu at 7:25:30:737 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {5 } Elements removed: {}] }]} 


transition: output SEND(3, 3, 4) in automaton LCR(3) 
on tui.csail.mit.edu at 7:25:30:959 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[4 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {3 } Elements removed: {}] }]} 


transition: output SEND(7, 7, 0) in automaton LCR(7) 
on tui.csail.mit.edu at 7:25:30:989 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[0 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


Initialization starts (1) on condor.csail.mit.edu at 7:25:31:086 
Modified state variables: 
P — [pending: (), status: idle] 
RM — Map{} 
SM — Map{} 
rank — null 
size — null 
Initialization ends 
transition: input vote() in automaton LCR(1) 
on condor.csail.mit.edu at 7:25:31:347 
Modified state variables: 
P — [pending: (1), status: voting] 
RM — Map{[0 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[2 -> [status: idle, toSend: {}, sent: {}, handles: {}]] } 
rank — 1 
size — 8 


transition: output RECEIVE(7, 7, 0) in automaton LCR(0) 
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on loon.csail.mit.edu at 7:25:31:445 
Modified state variables: 
P — Tuple, modified fields: {[pending -> (7)] } 
SM — Map, modified entries: {[1 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {0 }] [sent -> Sequence, elements 
added: {0 } Elements removed: {}] }]} 


transition: output SEND(7, 0, 1) in automaton LCR(0) 
on loon.csail.mit.edu at 7:25:31:448 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[1 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


transition: output SEND(1, 1, 2) in automaton LCR(1) 
on condor.csail.mit.edu at 7:25:31:489 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[2 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {1 } Elements removed: {}] }]} 


transition: output RECEIVE(0, 0, 1) in automaton LCR(1) 
on condor.csail.mit.edu at 7:25:32:205 
Modified state variables: 
SM — Map, modified entries: {[2 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {1 }] [sent -> Sequence, elements 
added: {1 } Elements removed: {}] }]} 


transition: output RECEIVE(1, 1, 2) in automaton LCR(2) 
on parrot.csail.mit.edu at 7:25:32:243 
Modified state variables: 
SM — Map, modified entries: {[3 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {2 }] [sent -> Sequence, elements 
added: {2 } Elements removed: {}] }]} 


transition: output RECEIVE(6, 6, 7) in automaton LCR(7) 
on tui.csail.mit.edu at 7:25:32:274 
Modified state variables: 
SM — Map, modified entries: {[0 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {7 }] [sent -> Sequence, elements 
added: {7 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 0, 1) in automaton LCR(1) 
on condor.csail.mit.edu at 7:25:32:325 

Modified state variables: 

P — Tuple, modified fields: {[pending -> (7)] } 


transition: output RECEIVE(5, 5, 6) in automaton LCR(6) 
on parrot.csail.mit.edu at 7:25:32:437 
transition: output SEND(7, 1, 2) in automaton LCR(1) 
on condor.csail.mit.edu at 7:25:32:466 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[2 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 
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Modified state variables: 

SM — Map, modified entries: {[7 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {6 }] [sent -> Sequence, elements 
added: {6 } Elements removed: {}] }]} 


transition: output RECEIVE(2, 2, 3) in automaton LCR(3) 
on tui.csail.mit.edu at 7:25:32:584 
Modified state variables: 
SM — Map, modified entries: {[4 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {3 }] [sent -> Sequence, elements 
added: {3 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 1, 2) in automaton LCR(2) 
on parrot.csail.mit.edu at 7:25:32:777 

Modified state variables: 

P — Tuple, modified fields: {[pending -> (7)] } 


transition: output SEND(7, 2, 3) in automaton LCR(2) 
on parrot.csail.mit.edu at 7:25:32:997 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[3 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 2, 3) in automaton LCR(3) 
on tui.csail.mit.edu at 7:25:33:121 

Modified state variables: 

P — Tuple, modified fields: {[pending -> (7)] } 


transition: output SEND(7, 3, 4) in automaton LCR(3) 
on tui.csail.mit.edu at 7:25:33:135 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[4 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


Initialization starts (4) on loon.csail.mit.edu at 7:25:36:355 
Modified state variables: 
P — [pending: (), status: idle] 
RM — Map{} 
SM — Map{} 
rank — null 
size — null 
Initialization ends 
transition: input vote() in automaton LCR (4) 
on loon.csail.mit.edu at 7:25:36:365 
Modified state variables: 
P — [pending: (4), status: voting] 
RM — Map{[3 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[5 -> [status: idle, toSend: {}, sent: {}, handles: {}]] } 
rank — 4 
size — 8 


transition: output SEND(4, 4, 5) in automaton LCR (4) 


on loon.csail.mit.edu at 7:25:36:369 
Modified state variables: 
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P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {4 } Elements removed: {}] }]} 


transition: output RECEIVE(3, 3, 4) in automaton LCR(4) 
on loon.csail.mit.edu at 7:25:36:553 
Modified state variables: 
SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {4 }] [sent -> Sequence, elements 
added: {4 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 3, 4) in automaton LCR(4) 
on loon.csail.mit.edu at 7:25:36:557 

Modified state variables: 

P — Tuple, modified fields: {[pending -> (7)] } 


transition: output SEND(7, 4, 5) in automaton LCR (4) 
on loon.csail.mit.edu at 7:25:36:559 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


transition: output RECEIVE(4, 4, 5) in automaton LCR(5) 
on condor.csail.mit.edu at 7:25:37:074 
Modified state variables: 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {} Elements removed: {5 }] [sent -> Sequence, elements 
added: {5 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 4, 5) in automaton LCR(5) 
on condor.csail.mit.edu at 7:25:37:274 

Modified state variables: 

P — Tuple, modified fields: {[pending -> (7)] } 


transition: output SEND(7, 5, 6) in automaton LCR(5) 
on condor.csail.mit.edu at 7:25:37:280 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 5, 6) in automaton LCR(6) 
on parrot.csail.mit.edu at 7:25:37:755 

Modified state variables: 

P — Tuple, modified fields: {[pending -> (7)] } 


transition: output SEND(7, 6, 7) in automaton LCR(6) 
on parrot.csail.mit.edu at 7:25:37:770 
Modified state variables: 
P — Tuple, modified fields: {[pending -> ()] } 
SM — Map, modified entries: {[7 -> Tuple, modified fields: {[toSend -> 
Sequence, elements added: {7 } Elements removed: {}] }]} 


transition: output RECEIVE(7, 6, 7) in automaton LCR(7) 
on tui.csail.mit.edu at 7:25:37:872 
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Modified state variables: 
P — Tuple, modified fields: {[status -> elected] } 


transition: output leader(7) in automaton LCR(7) 
on tui.csail.mit.edu at 7:25:37:874 


Modified state variables: 
P — Tuple, modified fields: {[status -> announced] } 


E.2 Asynchronous Spanning Tree on 16 nodes 


Complete trace 


Begin initialization 


Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

transition: input initialize() in automaton sTreeNode(6) on 
tui.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 2 5 7), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i—- 6 


Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

transition: input initialize() in automaton sTreeNode(0) on 
loon.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 4), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i—- 0 


transition: output SEND(search, 0, 4) in automaton sTreeNode(0) on 


loon.csail.mit.edu 
Modified state variables: 
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P — [nbrs: (1 4), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 

Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i — null 

End initialization 

transition: output SEND(search, 0, 1) in automaton sTreeNode(0) on 


csail.mit.edu 

Modified state variables: 
(1 4), 
ioa.runtime.adt.MapSort] 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


loon. 


P — [nbrs: parent: -1, 


transition: input initialize () 
parrot.csail.mit.edu 
Begin initialization 
Modified state variables: 
Modified state variables: 


reported: false, send: 


in automaton sTreeNode(11) on 


P — [nbrs: (10 15 7), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i- 11 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 


RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i — null 

End initialization 

transition: input initialize () 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (13 8), parent: 
ioa.runtime.adt.MapSort] 
.adt.MapSort 
.adt.MapSort 


4 


RM — ioa.runtime 
SM — ioa.runtime 
io— 12 
transition: input initialize () 
csail.mit.edu 
transition: input 
parrot.csail.mit.edu 
Begin initialization 


nene. 
initialize () 


in automaton sTreeNode (12) on 


reported: false, send: 


> 


in automaton sTreeNode(10) on 
in automaton sTreeNode(5) on 


Begin initialization 
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Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i — null 

End initialization 

transition: input initialize() in automaton sTreeNode(1i) on 
blackbird.csail.mit.edu 

Begin initialization 

Modified state variables: 

P — [nbrs: (0 2 5), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


RM — ioa.runtime.adt.MapSort Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 
RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i — null 

End initialization 


SM — ioa.runtime.adt.MapSort 
ied 


Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

Begin initialization 

transition: input initialize() in automaton sTreeNode (4) on 
parrot.csail.mit.edu 

transition: input initialize() in automaton sTreeNode(7) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i — null 

End initialization 

Modified state variables: 

P — [nbrs: (11 3 6), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 
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SM — ioa.runtime.adt.MapSort 
a oe 


transition: input initialize() in automaton sTreeNode(13) on 
tui.csail.mit.edu 

transition: input initialize() in automaton sTreeNode(9) on 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (12 14 9), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— 13 


Begin initialization 

Modified state variables: 

Modified state variables: 

P — [nbrs: (10 13 5 8), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 


SM — ioa.runtime.adt.MapSort 


RM — ioa.runtime.adt.MapSort 
i- 9 SM — ioa.runtime.adt.MapSort 


io— null 


End initialization 

transition: input initialize() in automaton sTreeNode(15) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (11 14), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— 15 


Modified state variables: 

P — [nbrs: (1 4 6 9), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io—- 5 


SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

transition: input initialize() in automaton sTreeNode(14) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 15), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 
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io— 14 


Modified state variables: 

P — [nbrs: (0 5 8), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i—- 4 


Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

transition: input initialize() in automaton sTreeNode(8) on 
blackbird.csail.mit.edu 

transition: output RECEIVE(search, 4, 0) in automaton sTreeNode(4) on 
parrot.csail.mit.edu 

transition: output RECEIVE(search, 1, 0) in automaton sTreeNode(1) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (0 2 5), parent: 0, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

Modified state variables: 

P — [nbrs: (0 5 8), parent: 0, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(0O) in automaton sTreeNode(4) on 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (0 5 8), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 

transition: output PARENT(0O) in automaton sTreeNode(1) on 
blackbird.csail.mit.edu 

Modified state variables: 


transition: output SEND(search, 4, 8) in automaton sTreeNode(4) on 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (0 2 5), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 


transition: output SEND(search, 1, 2) in automaton sTreeNode(1) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (0 2 5), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


P — [nbrs: (0 5 8), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 
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SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [nbrs: (12 4 9), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i— 8 


transition: output SEND(search, 1, 5) in automaton sTreeNode(1) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (0 2 5), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 4, 5) in automaton sTreeNode(4) on 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (0 5 8), parent: 0, reported: true, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 5, 1) in automaton sTreeNode(5) on 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 4 6 9), parent: 1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(1) in automaton sTreeNode(5) on 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 4 6 9), parent: 1, reported: true, send: 
ioa.runtime.adt.MapSort] 


transition: output SEND(search, 5, 4) in automaton sTreeNode(5) on 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 4 6 9), parent: 1, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [nbrs: (11 14 6 9), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i-— 10 


transition: output RECEIVE(search, 5, 4) in automaton sTreeNode(5) on 
parrot.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 
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SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 4 6 9), 
ioa.runtime.adt.MapSort] 


parent: 
SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 4 6 9), 
ioa.runtime.adt.MapSort] 


parent: 


RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 5 8), 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT (5) 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 5 8), 
ioa.runtime.adt.MapSort] 


in 


transition: output SEND(search, 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 5 8), 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 5 8), 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 5 8), 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: 
blackbird.csail.mit.edu 
transition: 


1, 


1, 


output RECEIVE(search, 


parent: 


parent: 


parent: 


parent: 


parent: 


output RECEIVE(search, 


output RECEIVE(search, 


5, 9) in automaton sTreeNode(5) on 
reported: true, send: 

5, 6) in automaton sTreeNode(5) on 
reported: true, send: 
9, 5) in automaton sTreeNode(9) on 


5, reported: false, send: 


automaton sTreeNode(9) on 


5, reported: true, send: 
9, 13) in automaton sTreeNode(9) on 
5, reported: true, send: 
9, 8) in automaton sTreeNode(9) on 
5, reported: true, send: 
9, 10) in automaton sTreeNode(9) on 
5, reported: true, send: 
8, 4) in automaton sTreeNode(8) on 


4, 5) in automaton sTreeNode (4) on 
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parrot.csail.mit.edu 
Modified state variables: 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [nbrs: (12 4 9), parent: 4, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(4) in automaton sTreeNode(8) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (12 4 9), parent: 4, reported: true, send: 
ioa.runtime.adt.MapSort] 


transition: output SEND(search, 8, 9) in automaton sTreeNode(8) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (12 4 9), parent: 4, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 8, 9) in automaton sTreeNode(8) on 
blackbird.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 8, 12) in automaton sTreeNode(8) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (12 4 9), parent: 4, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 9, 8) in automaton sTreeNode(9) on 
condor.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 10, 9) in automaton sTreeNode (10) on 
nene.csail.mit.edu 

Modified state variables: 

P — [nbrs: (11 14 6 9), parent: 9, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(9) in automaton sTreeNode(10) on 
nene.csail.mit.edu 

Modified state variables: 

P — [nbrs: (11 14 6 9), parent: 9, reported: true, send: 
ioa.runtime.adt.MapSort] 


transition: output SEND(search, 10, 6) in automaton sTreeNode(10) on 
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nene.csail.mit.edu 

Modified state variables: 
(11 14 6 9), 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


P — [nbrs: 


transition: output SEND(search, 
nene.csail.mit.edu 
Modified state variables: 
P — [nbrs: (11 14 6 9), 
ioa.runtime.adt.MapSort] 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 


nene.csail.mit.edu 

Modified state variables: 

P — [nbrs: (11 14 6 9), parent: 
ioa.runtime.adt.MapSort] 


RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: 
csail.mit.edu 
Modified state variables: 
(10 2 5 7), 
.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


tui. 
P — [nbrs: parent: 
ioa 


transition: output PARENT(5) in 
Modified state variables: 
(10 2 5 7), 


.runtime.adt.MapSort] 


P — [nbrs: parent: 


ioa 
transition: output SEND(search, 
csail.mit.edu 
Modified state variables: 
(10 2 5 7), 
.runtime.adt.MapSort] 
SM — ioa.runtime.adt.MapSort 


tui. 


P — [nbrs: parent: 


ioa 


transition: 
csail.mit.edu 


output SEND(search, 

tui. 

Modified state variables: 

(10 2 5 7), 
.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


P — [nbrs: parent: 


ioa 


transition: output SEND(search, 
csail.mit.edu 

Modified state variables: 

(10 25 7), 
runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


SM — ioa.runtime.adt.MapSort 


tui. 


P — [nbrs: parent: 


ioa. 


parent: 


parent: 


10, 


output RECEIVE(search, 


5, 


automaton sTreeNode(6) on tui.csail.mit.edu 


5, 


reported: 


1) 


reported: 


14) 


reported: 


6, 


reported: 


reported: 


2) 


reported: 


7) 


reported: 


10) 


reported: 
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true, send: 


in automaton sTreeNode(10) on 


true, send: 


in automaton sTreeNode(10) on 


true, send: 


5) in automaton sTreeNode(6) on 


send: 


false, 


send: 


true, 


sTreeNode(6) on 


in automaton 


true, send: 


in automaton sTreeNode(6) on 


true, send: 


in automaton sTreeNode(6) on 


true, send: 


tui. 


transition: output RECEIVE(search, 6, 10) in automaton sTreeNode(6) on 
csail.mit.edu 
transition: output RECEIVE(search, 10, 6) in automaton sTreeNode (10) on 


nene.csail.mit.edu 


tui. 


ioa. 


tui. 


ioa. 


tui. 


ioa 


Modified state variables: 
Modified state variables: RM — ioa.runtime.adt.MapSort 


RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 13, 9) in automaton sTreeNode (13) on 
csail.mit.edu 

Modified state variables: 

P — [nbrs: (12 14 9), parent: 9, reported: false, send: 
runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(9) in automaton sTreeNode(13) on 
csail.mit.edu 

Modified state variables: 

P — [nbrs: (12 14 9), parent: 9, reported: true, send: 
runtime.adt.MapSort] 


transition: output SEND(search, 13, 14) in automaton sTreeNode(13) on 
csail.mit.edu 

Modified state variables: 

P — [nbrs: (12 14 9), parent: 9, reported: true, send: 


.runtime.adt.MapSort] 


SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 12, 8) in automaton sTreeNode (12) on 


parrot.csail.mit.edu 


ioa. 


Modified state variables: 

P — [nbrs: (13 8), parent: 8, reported: false, send: 
runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(8) in automaton sTreeNode(12) on 


parrot.csail.mit.edu 


ioa. 


Modified state variables: 
P — [nbrs: (13 8), parent: 8, reported: true, send: 
runtime.adt.MapSort] 


transition: output SEND(search, 12, 13) in automaton sTreeNode(12) on 


parrot.csail.mit.edu 


ioa. 


tui. 


Modified state variables: 
P — [nbrs: (13 8), parent: 8, reported: true, send: 
runtime.adt.MapSort] 
SM — ioa.runtime.adt.MapSort 

transition: output SEND(search, 13, 12) in automaton sTreeNode (13) 
csail.mit.edu 
Modified state variables: 
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on 


P — [nbrs: (12 14 9), parent: 9, reported: true, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(search, 13, 12) in automaton sTreeNode(13) on 


tui.csail.mit.edu 
Modified state variables: 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 
parrot.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(search, 
parrot.csail.mit.edu 

Modified state variables: 
(10 15 7), parent: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


P — [nbrs: 10, 


transition: output PARENT (10) 
parrot.csail.mit.edu 


Modified state variables: 


P — [nbrs: (10 15 7), parent: 10, 
ioa.runtime.adt.MapSort] 

transition: output SEND(search, 11, 
parrot.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 15 7), parent: 10, 


ioa.runtime.adt.MapSort] 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 
blackbird.csail.mit.edu 

Modified state variables: 
(11 3 6), 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


P — [nbrs: parent: 6, 


transition: output PARENT (6) 
blackbird.csail.mit.edu 


Modified state variables: 


P — [nbrs: (11 3 6), parent: 6, reported: 
ioa.runtime.adt.MapSort] 
transition: output SEND(search, 7, 3) 


blackbird.csail.mit.edu 

Modified state variables: 
(11 3 6), parent: 
ioa.runtime.adt.MapSort] 


P — [nbrs: 6, 


12, 13) in automaton sTreeNode (12) on 
11, 10) in automaton sTreeNode(1ii) on 
reported: false, send: 


reported: 


15) 


reported: 


7, 
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6) 


reported: 


in automaton sTreeNode(11) on 


true, send: 


in automaton sTreeNode(1i1) on 


true, send: 


in automaton sTreeNode(7) on 


false, send: 


in automaton sTreeNode(7) on 


true, send: 


in automaton sTreeNode(7) on 


reported: 


true, send: 


SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 7, 11) in automaton sTreeNode(7) on 


blackbird.csail.mit.edu 


Modified state variables: 
P — [nbrs: (11 3 6), parent: 6, reported: true, send: 


ioa.runtime.adt.MapSort] 


nene. 


RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 
RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

transition: input initialize() in automaton sTreeNode(3) on 

csail.mit.edu 

transition: output SEND(search, 11, 7) in automaton sTreeNode(1i1) on 


parrot.csail.mit.edu 


Modified state variables: 
P — [nbrs: (10 15 7), parent: 10, reported: true, send: 


ioa.runtime.adt.MapSort] 


RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 11, 7) in automaton sTreeNode(11) on 


parrot.csail.mit.edu 


Modified state variables: 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 7, 11) in automaton sTreeNode(7) on 


blackbird.csail.mit.edu 


Modified state variables: 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 14, 10) in automaton sTreeNode(14) on 


blackbird.csail.mit.edu 


Modified state variables: 
P — [nbrs: (10 13 15), parent: 10, reported: false, send: 


ioa.runtime.adt.MapSort] 


RM — ioa.runtime.adt.MapSort 


transition: output PARENT(10) in automaton sTreeNode(14) on 


blackbird.csail.mit.edu 


Modified state variables: 
P — [nbrs: (10 13 15), parent: 10, reported: true, send: 


ioa.runtime.adt.MapSort] 


black 


transition: output SEND(search, 14, 15) in automaton sTreeNode(14) on 
bird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 15), parent: 10, reported: true, send: 
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ioa.runtime.adt.MapSort] 
SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [nbrs: (2 7), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— 3 


transition: output RECEIVE(search, 3, 7) in automaton sTreeNode(3) on 
nene.csail.mit.edu 

Modified state variables: 

P — [nbrs: (2 7), parent: 7, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(7) in automaton sTreeNode(3) on 
nene.csail.mit.edu 

Modified state variables: 

P — [nbrs: (2 7), parent: 7, reported: true, send: 
ioa.runtime.adt.MapSort] 


transition: output SEND(search, 3, 2) in automaton sTreeNode(3) on 
nene.csail.mit.edu 

Modified state variables: 

P — [nbrs: (2 7), parent: 7, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 14, 13) in automaton sTreeNode(14) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (10 13 15), parent: 10, reported: true, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 14, 13) in automaton sTreeNode (14) 
blackbird.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 13, 14) in automaton sTreeNode (13) 
tui.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 15, 11) in automaton sTreeNode (15) 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (11 14), parent: 11, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 
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on 


on 


on 


transition: output PARENT(11) in automaton sTreeNode(15) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (11 14), parent: 11, reported: true, send: 
ioa.runtime.adt.MapSort] 


transition: output SEND(search, 15, 14) in automaton sTreeNode(15) on 
blackbird.csail.mit.edu 

Modified state variables: 

P — [nbrs: (11 14), parent: 11, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 15, 14) in automaton sTreeNode(15) on 
blackbird.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(search, 14, 15) in automaton sTreeNode(14) on 
blackbird.csail.mit.edu 


Modified state variables: 
RM — ioa.runtime.adt.MapSort 


Begin initialization 

Modified state variables: 

P — [nbrs: (), parent: 87, reported: true, send: ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— null 

End initialization 

transition: input initialize() in automaton sTreeNode(2) on 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 3 6), parent: -1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i—- 2 


transition: output RECEIVE(search, 2, 1) in automaton sTreeNode(2) on 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 3 6), parent: 1, reported: false, send: 
ioa.runtime.adt.MapSort] 

RM — ioa.runtime.adt.MapSort 


transition: output PARENT(1) in automaton sTreeNode(2) on 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 3 6), parent: 1, reported: true, send: 
ioa.runtime.adt.MapSort] 


transition: output SEND(search, 2, 3) in automaton sTreeNode(2) on 
condor.csail.mit.edu 
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Modified state variables: 

P — [nbrs: (1 3 6), parent: 1, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 2, 3) in automaton sTreeNode(2) on 
condor.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(search, 2, 6) in automaton sTreeNode(2) on 
condor.csail.mit.edu 

Modified state variables: 

P — [nbrs: (1 3 6), parent: 1, reported: true, send: 
ioa.runtime.adt.MapSort] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 6, 2) in automaton sTreeNode(6) on 
tui.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 2, 6) in automaton sTreeNode(2) on 
condor.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(search, 3, 2) in automaton sTreeNode(3) on 
nene.csail.mit.edu 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


E.3 Asynchronous Broadcast Convergecast on 16 nodes 


Complete trace 


Begin initialization 

Modified state variables: 

P — [val: 87, acked: (), nbrs: (), parent: 87, reported: true, send: 
ioa.runtime.adt.MapSort, 

temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i 0 

End initialization 

transition: input initialize() in automaton bcastNode 

Modified state variables: 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 1, 0) in automaton 


bcastNode 


P — [val: -1, acked: (), nbrs: (0 5 8), parent: -1, reported: false, 
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send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 
RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i- 4 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 2 5), parent: 0, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 1, 2) in automaton 


bcastNode 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 2 5), parent: 0, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 1, 5) in automaton 


bcastNode 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 2 5), parent: 0, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


Begin initialization 

Modified state variables: 

P — [val: 87, acked: (), nbrs: (), parent: 87, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

7 fee a 0) 

End initialization 

transition: input initialize() in automaton bcastNode 

Modified state variables: 

P — [val: -1, acked: (), nbrs: (10 13 15), parent: -1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— 14 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 4, 0) in automaton 


bcastNode 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 5, 1) in automaton 


bcastNode 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 5, 4) in automaton 


bcastNode 


send: 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 4 6 9), parent: 1, reported: false, 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 
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Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 5 8), parent: 0, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 4, 8) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 5 8), parent: 0, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 5, 9) in automaton 
bcastNode 

transition: output SEND(msg([kind: bcast, w: 99]), 4, 5) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 5 8), parent: 0, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 4, 5) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 5 8), parent: 0, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 4, 5) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (0 5 8), parent: 0, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 5, 6) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 5, 4) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 4 6 9), parent: 1, reported: false, 
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send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 5, 4) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 5, 4) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (4), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 8, 4) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (12 4 9), parent: 4, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 8, 9) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (12 4 9), parent: 4, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 4, 5) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5), nbrs: (0 5 8), parent: 0, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 2, 1) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 3 6), parent: 1, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 2, 3) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 3 6), parent: 1, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 8, 12) in automaton 


bcastNode 
Modified state variables: 
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P — [val: 99, acked: (), nbrs: (12 4 9), parent: 4, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


Begin initialization 

Modified state variables: 

P — [val: 87, acked: (), nbrs: (), parent: 87, reported: true, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

io— 0 

End initialization 

transition: input initialize() in automaton bcastNode 

transition: output SEND(msg([kind: bcast, w: 99]), 2, 6) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (1 3 6), parent: 1, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [val: -1, acked: (), nbrs: (10 2 5 7), parent: -1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i—- 6 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 6, 2) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 6, 10) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 3, 2) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (2 7), parent: 2, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 3, 7) in automaton 
bcastNode 

transition: output SEND(msg([kind: bcast, w: 99]), 6, 7) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 2 5 7), parent: 2, reported: false, 
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send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 6, 5) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 6, 5) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 6, 5) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 12, 8) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (13 8), parent: 8, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 12, 13) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (13 8), parent: 8, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 5, 6) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (4), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 5, 6) in automaton bcastNode 

Modified state variables: 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (2 7), parent: 2, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 

P — [val: 99, acked: (4), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87 ]] 
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RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 5, 6) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (4 6), nbrs: (1 4 6 9), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 6, 5) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 13, 12) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (12 14 9), parent: 12, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 13, 9) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (12 14 9), parent: 12, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 13, 14) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (12 14 9), parent: 12, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 7, 3) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 3 6), parent: 3, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 7, 11) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 3 6), parent: 3, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 7, 6) in automaton 
bcastNode 
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Modified state variables: 


P — [val: 99, acked: (), nbrs: (11 3 6), parent: 3, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(msg([kind: bcast, w: 99]), 7, 6) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 3 6), parent: 3, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output SEND(kind(ack), 7, 6) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 3 6), parent: 3, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(msg([kind: bcast, w: 99]), 6, 7) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (5), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

transition: output SEND(kind(ack), 6, 7) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 6, 7) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5 7), nbrs: (10 2 5 7), parent: 2, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 7, 6) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (6), nbrs: (11 3 6), parent: 3, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(msg([kind: bcast, w: 99]), 11, 7) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 15 7), parent: 7, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


RM — ioa.runtime.adt.MapSort 
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transition: output SEND(msg([kind: bcast, w: 99]), 11, 10) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 15 7), parent: 7, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 14, 13) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 14, 10) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 11, 15) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 15 7), parent: 7, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 14, 15) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 11, 10) in automaton 
bcastNode 

transition: output RECEIVE(msg([kind: bcast, w: 99]), 10, 6) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 10, 11) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 10, 11) in automaton 


bcastNode 
Modified state variables: 
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P — [val: 99, acked: (), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 10, 14) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 10, 14) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 10, 9) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 15 7), parent: 7, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 11, 10) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 15 7), parent: 7, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 14, 10) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 14, 10) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 
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transition: output SEND(kind(ack), 10, 11) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 11, 10) in automaton bcastNode 

transition: output RECEIVE(msg([kind: bcast, w: 99]), 15, 11) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14), parent: 11, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 15, 14) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14), parent: 11, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 15, 14) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14), parent: 11, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 15, 14) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (11 14), parent: 11, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [val: 99, acked: (10), nbrs: (10 15 7), parent: 7, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 10, 11) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (11), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 10, 14) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (11), nbrs: (11 14 6 9), parent: 6, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 
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transition: output RECEIVE(kind(ack), 10, 14) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (11 14), nbrs: (11 14 6 9), parent: 6, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 14, 10) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 14, 15) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (10), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 14, 15) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10), nbrs: (10 13 15), parent: 13, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 14, 15) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 15), nbrs: (10 13 15), parent: 13, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


Begin initialization 

Modified state variables: 

P — [val: 87, acked: (), nbrs: (), parent: 87, reported: true, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i—- 0 

End initialization 

transition: input initialize() in automaton bcastNode 

Modified state variables: 

P — [val: -1, acked: (), nbrs: (10 13 5 8), parent: -1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

i- 9 


transition: internal report(14) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 15), nbrs: (10 13 15), parent: 13, reported: 
true, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 
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transition: output SEND(kind(ack), 14, 13) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 15), nbrs: (10 13 15), parent: 13, reported: 
true, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 13, 14) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14), nbrs: (12 14 9), parent: 12, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 15, 14) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14), nbrs: (11 14), parent: 11, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: internal report(15) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14), nbrs: (11 14), parent: 11, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

transition: output SEND(kind(ack), 15, 11) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14), nbrs: (11 14), parent: 11, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 11, 15) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 15), nbrs: (10 15 7), parent: 7, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

transition: internal report(11) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 15), nbrs: (10 15 7), parent: 7, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

transition: output SEND(kind(ack), 11, 7) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 15), nbrs: (10 15 7), parent: 7, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 7, 11) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (11 6), nbrs: (11 3 6), parent: 3, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


RM — ioa.runtime.adt.MapSort 


transition: internal report (7) 
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in automaton bcastNode 


Modified state variables: 
P — [val: 99, acked: (11 6), nbrs: (11 3 6), parent: 3, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 7, 3) in automaton bcastNode 

transition: output RECEIVE(kind(ack), 3, 7) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (7), nbrs: (2 7), parent: 2, reported: false, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: internal report(3) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (7), nbrs: (2 7), parent: 2, reported: true, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 3, 2) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (7), nbrs: (2 7), parent: 2, reported: true, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


Modified state variables: 

P — [val: 99, acked: (11 6), nbrs: (11 3 6), parent: 3, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 2, 3) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (3), nbrs: (1 3 6), parent: 1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort transition: output RECEIVE (msg([kind: 
beast, w: 99]), 9, 5) in automaton bcastNode 


Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 9, 13) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 9, 8) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 
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SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 13, 9) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (14), nbrs: (12 14 9), parent: 12, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 13, 9) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14), nbrs: (12 14 9), parent: 12, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 9, 8) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(msg([kind: bcast, w: 99]), 9, 10) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 10, 9) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (11 14), nbrs: (11 14 6 9), parent: 6, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(msg([kind: bcast, w: 99]), 9, 10) in automaton 
bcastNode 


Modified state variables: 

transition: output SEND(kind(ack), 10, 9) in automaton bcastNode 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

Modified state variables: 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort P — [val: 99, acked: (11 14), nbrs: 
(11 14 6 9), parent: 6, reported: false, send: ioa.runtime.adt.MapSort, temp: 
[kind: bcast, w: 87]] 


RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 9, 13) in automaton 
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bcastNode 
Modified state variables: 


P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

transition: output SEND(kind(ack), 9, 8) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 

transition: output SEND(kind(ack), 9, 10) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 10, 9) in automaton bcastNode 

Modified state variables: 

transition: output RECEIVE(kind(ack), 9, 10) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

P — [val: 99, acked: (11 14 9), nbrs: (11 14 6 9), parent: 6, reported: 
false, send: ioa.runtime.adt.MapSort, 

temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort SM — ioa.runtime.adt.MapSort 

transition: internal report(10) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (11 14 9), nbrs: (11 14 6 9), parent: 6, reported: 
true, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

transition: output SEND(kind(ack), 10, 6) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (11 14 9), nbrs: (11 14 6 9), parent: 6, reported: 
true, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 

transition: output SEND(kind(ack), 9, 13) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10), nbrs: (10 13 5 8), parent: 5, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 

transition: output RECEIVE(kind(ack), 9, 13) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 13), nbrs: (10 13 5 8), parent: 5, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 
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RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 13, 9) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14 9), nbrs: (12 14 9), parent: 12, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: internal report(13) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14 9), nbrs: (12 14 9), parent: 12, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 13, 12) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (14 9), nbrs: (12 14 9), parent: 12, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 12, 13) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (13), nbrs: (13 8), parent: 8, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: internal report(12) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (13), nbrs: (13 8), parent: 8, reported: true, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 12, 8) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (13), nbrs: (13 8), parent: 8, reported: true, send: 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(msg([kind: bcast, w: 99]), 8, 9) in automaton 
bcastNode 

Modified state variables: 

P — [val: 99, acked: (), nbrs: (12 4 9), parent: 4, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output SEND(kind(ack), 8, 9) in automaton bcastNode 

Modified state variables: 

transition: output RECEIVE(kind(ack), 6, 10) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 5 7), nbrs: (10 2 5 7), parent: 2, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 
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true, 


true, 


send: 


send: 


send: 


send: 


send: 


transition: internal report(6) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 5 7), nbrs: (10 2 5 7), parent: 2, reported: 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 6, 2) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (10 5 7), nbrs: (10 2 5 7), parent: 2, reported: 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 2, 6) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (3 6), nbrs: (1 3 6), parent: 1, reported: false, 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: internal report(2) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (3 6), nbrs: (1 3 6), parent: 1, reported: true, 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 2, 1) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (3 6), nbrs: (1 3 6), parent: 1, reported: true, 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 1, 2) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (2), nbrs: (0 2 5), parent: 0, reported: false, 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


P — [val: 99, acked: (), nbrs: (12 4 9), parent: 4, reported: false, 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 9, 8) in automaton bcastNode 
Modified state variables: 

transition: output RECEIVE(kind(ack), 8, 9) in automaton bcastNode 

P — [val: 99, acked: (10 13 8), nbrs: (10 13 5 8), parent: 5, reported: 


false, send: ioa.runtime.adt.MapSort, 


send: 


temp: [kind: bcast, w: 87]] 
RM — ioa.runtime.adt.MapSort 


Modified state variables: 

transition: internal report(9) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (9), nbrs: (12 4 9), parent: 4, reported: false, 
ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 
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SM — ioa.runtime.adt.MapSort 
P — [val: 99, acked: (10 13 8), nbrs: (10 13 5 8), parent: 5, reported: 
true, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 9, 5) in automaton bcastNode 


Modified state variables: 

P — [val: 99, acked: (10 13 8), nbrs: (10 13 5 8), parent: 5, reported: 
true, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 8, 12) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (12 9), nbrs: (12 4 9), parent: 4, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: internal report(8) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (12 9), nbrs: (12 4 9), parent: 4, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 8, 4) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (12 9), nbrs: (12 4 9), parent: 4, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 5, 9) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (4 6 9), nbrs: (1 4 6 9), parent: 1, reported: 
false, send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: internal report(5) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (4 6 9), nbrs: (1 4 6 9), parent: 1, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 5, 1) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (4 6 9), nbrs: (1 4 6 9), parent: 1, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 1, 5) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (2 5), nbrs: (0 2 5), parent: 0, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: internal report(1) in automaton bcastNode 
Modified state variables: 
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P — [val: 99, acked: (2 5), nbrs: (0 2 5), parent: 0, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 1, 0) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (2 5), nbrs: (0 2 5), parent: 0, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 4, 8) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5 8), nbrs: (0 5 8), parent: 0, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 


transition: internal report(4) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5 8), nbrs: (0 5 8), parent: 0, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 


transition: output SEND(kind(ack), 4, 0) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (5 8), nbrs: (0 5 8), parent: 0, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 87]] 

RM — ioa.runtime.adt.MapSort 

SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 0, 1) in automaton bcastNode 
Modified state variables: 


P — [val: 99, acked: (1), nbrs: (1 4), parent: -1, reported: false, send: 


ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 99]] 
RM — ioa.runtime.adt.MapSort 
SM — ioa.runtime.adt.MapSort 


transition: output RECEIVE(kind(ack), 0, 4) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (1 4), nbrs: (1 4), parent: -1, reported: false, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 99]] 

RM — ioa.runtime.adt.MapSort 


transition: internal report(0) in automaton bcastNode 

Modified state variables: 

P — [val: 99, acked: (1 4), nbrs: (1 4), parent: -1, reported: true, 
send: ioa.runtime.adt.MapSort, temp: [kind: bcast, w: 99]] 


EK.4 Spanning Tree to Leader Election on 16 nodes 


Complete trace 


Trace of sTreeLeader 


Number of machines: 5 
Number of nodes: 16 
Duration: 9 sec 
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Number of messages exchanged: 16 


Initialization starts (15) on loon.csail.mit.edu at 9:29:05:372 
Modified state variables: 
P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: Map{}] 
RM — Map{} 
SM — Map{} 
fe BT 
k — 87 
rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 
transition: output SEND(elect, 15, 11) in automaton sTreeLeader (15) 
on loon.csail.mit.edu at 9:29:05:388 
Modified state variables: 
P — [nbrs: (11), receivedElect: (), sentElect: (), status: idle, send: Map{[11 -> {}] }] 
RM — Map{[11 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[11 -> [status: idle, toSend: {elect}, sent: {}, handles: {}]] } 
ee 
k — 11 
rank — 15 
tempNbrs — () 
tempNbrs2 — () 


Initialization starts (0) on loon.csail.mit.edu at 9:29:05:847 
Modified state variables: 
P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: Map{}] 
RM — Map{} 
SM — Map{} 
ee ec 
k — 87 
rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 
transition: output SEND(elect, 0, 1) in automaton sTreeLeader (0) 
on loon.csail.mit.edu at 9:29:05:860 
Modified state variables: 
P — [nbrs: (1), receivedElect: (), sentElect: (), status: idle, send: Map{[1 -> {}] }] 
RM — Map{[1 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[1 -> [status: idle, toSend: felect}, sent: {}, handles: {}]] } 
ji 
k — 1 
rank — 0 
tempNbrs — () 
tempNbrs2 — () 


Initialization starts (14) on tui.csail.mit.edu at 9:29:05:977 
Initialization starts (3) on parrot.csail.mit.edu at 9:29:05:978 

Modified state variables: 

Modified state variables: 

P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: Map{}] 
RM — Map{} 

SM — Map{} 
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j — 87 P — [nbrs: (), receivedElect: (), sentElect: (), status: 


k — 87 
RM — Mapf{} rank — null 


tempNbrs — () 
tempNbrs2 — () 
SM — Map{} Initialization ends 


j — 87 

k — 87 

rank — null 

tempNbrs — () 

tempNbrs2 — () 

Initialization ends 

Initialization starts (13) on parrot.csail.mit.edu at 9:29:06:210 
Modified state variables: 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

Initialization starts (8) on parrot.csail.mit.edu at 9:29:06:475 
Initialization starts (11) on drake.csail.mit.edu at 9:29:06:477 
Modified state variables: 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

SM — Map{} 

j — 87 

k — 87 

rank — null 

tempNbrs — () 

tempNbrs2 — () 
Initialization ends 
Modified state variables: 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

SM — Map{} 

4. = 87 

k — 87 

rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 
SM — Map{} 

j — 87 

k — 87 

rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 
transition: output SEND(elect, 13, 14) in automaton sTreeLeader (13) 


on parrot.csail.mit.edu at 9:29:06:796 


Modified state variables: 


idle, send: Map{}] 


Map {}] 


Map {}] 


Map {}] 


P — [nbrs: (14), receivedElect: (), sentElect: (), status: idle, send: Map{[14 -> {}] }] 


RM — Map{[14 -> [status: idle, toRecv: {}, ready: false]] } 

SM — Map{[14 -> [status: idle, toSend: {elect}, sent: {}, handles: 
j = 14 

k — 14 
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{}]] } 


rank — 13 
tempNbrs — () 
tempNbrs2 — () 


Initialization starts (1) on drake.csail.mit.edu at 9:29:07:059 
Initialization starts (2) on condor.csail.mit.edu at 9:29:07:062 
Modified state variables: 
Initialization starts (4) on tui.csail.mit.edu at 9:29:07:150 
Modified state variables: 
P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: Map{}] 
RM — Map{} 
SM — Map{} 
je Be 
k — 87 
rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 
transition: output RECEIVE(elect, 11, 15) in automaton sTreeLeader (11) 
on drake.csail.mit.edu at 9:29:07:180 
Modified state variables: 
P — [nbrs: (15 7), receivedElect: (15), sentElect: (7), status: idle, send: Map{[15 -> {}] 
RM — Map{[15 -> [status: idle, toRecv: {}, ready: false]] [7 -> [status: idle, toRecv: f{}, 
SM — Map{[15 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [7 -> [status: idle, tc 
j37 
k — 15 
rank — 11 
tempNbrs — () 
tempNbrs2 — () 


Initialization starts (6) on drake.csail.mit.edu at 9:29:07:304 
Modified state variables: 
transition: output SEND(elect, 4, 5) in automaton sTreeLeader (4) 
on tui.csail.mit.edu at 9:29:07:401 
Modified state variables: 
P — [nbrs: (5), receivedElect: (), sentElect: (), status: idle, send: Map{[5 -> {}] }] 
RM — Map{[5 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[5 -> [status: idle, toSend: felect}, sent: {}, handles: {}]] } 
jos 
k — 5 
rank — 4 
tempNbrs — () 
tempNbrs2 — () 


transition: output SEND(elect, 11, 7) in automaton sTreeLeader (11) 

on drake.csail.mit.edu at 9:29:07:485 
Modified state variables: 
P — Tuple, modified fields: {[send -> Map, modified entries: {[7 -> Sequence, elements adc 
SM — Map, modified entries: {[7 -> Tuple, modified fields: {[toSend -> Sequence, elements 
k — 7 


Initialization starts (7) on condor.csail.mit.edu at 9:29:07:260 

Modified state variables: 

P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: Map{}] 
RM — Map{} 

SM — Map{} 
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j — 87 

k — 87 

rank — null 

tempNbrs — () 

tempNbrs2 — () 

Initialization ends 

Initialization starts (12) on condor.csail.mit.edu at 9:29:07:712 
Modified state variables: 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

SM — Map{} 

j — 87 

k — 87 

rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

SM — Map{} 

ee 

k — 87 

rank — null 

tempNbrs — () 

tempNbrs2 — () 

Initialization ends 

Initialization starts (10) on loon.csail.mit.edu at 9:29:07:954 
Modified state variables: 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

SM — Map{} 

j — 87 

k — 87 

rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

SM — Map{} 

ie 

k — 87 

rank — null 

tempNbrs — () 

tempNbrs2 — () 

Initialization ends 

Initialization starts (9) on tui.csail.mit.edu at 9:29:08:081 
Modified state variables: 


P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: 


RM — Map{} 

SM — Map{} 
qs Be 

k — 87 

rank — null 
tempNbrs — () 
tempNbrs2 — () 
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Map {}] 


Map {}] 


Map {}] 


Map {}] 


Map {}] 


Initialization ends 
Modified state variables: 
P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: Map{}] 
RM — Map{} 
SM — Map{} 
i ae: 
k — 87 
rank — null 
tempNbrs — () 
tempNbrs2 — () 
Initialization ends 
transition: output SEND(elect, 12, 8) in automaton sTreeLeader (12) 
on condor.csail.mit.edu at 9:29:08:395 
Modified state variables: 
P — [nbrs: (8), receivedElect: (), sentElect: (), status: idle, send: Map{[8 -> {}] }] 
RM — Map{[8 -> [status: idle, toRecv: {}, ready: false]] } 
SM — Map{[8 -> [status: idle, toSend: felect}, sent: {}, handles: {}]] } 
j7 8 
k — 8 
rank — 12 
tempNbrs — () 
tempNbrs2 — () 


transition: output RECEIVE(elect, 1, 0) in automaton sTreeLeader (1) 
on drake.csail.mit.edu at 9:29:08:496 
transition: output RECEIVE(elect, 14, 13) in automaton sTreeLeader (14) 
on tui.csail.mit.edu at 9:29:08:528 
Modified state variables: 
P — [nbrs: (10 13), receivedElect: (13), sentElect: (10), status: idle, send: Map{[10 -> { 
RM — Map{[10 -> [status: idle, toRecv: {}, ready: false]] [13 -> [status: idle, toRecv: {]} 
SM — Map{[10 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [13 -> [status: idle, t 
ae 
k — 13 
rank — 14 
tempNbrs — () 
tempNbrs2 — () 


transition: output SEND(elect, 14, 10) in automaton sTreeLeader (14) 

on tui.csail.mit.edu at 9:29:08:536 
Modified state variables: 
P — Tuple, modified fields: {[send -> Map, modified entries: {[10 -> Sequence, elements ac 
SM — Map, modified entries: {[10 -> Tuple, modified fields: {[toSend -> Sequence, elements 
k — 10 
tempNbrs2 — (13) 


Modified state variables: 

P — [nbrs: (0 5), receivedElect: (0), sentElect: (5), status: idle, send: Map{[0 -> {}] [E 
RM — Map{[0 -> [status: idle, toRecv: {}, ready: false]] [5 -> [status: idle, toRecv: {}, 
SM — Map{[0 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [5 -> [status: idle, tos 
j > 0 

k — 0 

rank — 1 

tempNbrs — () 

tempNbrs2 — (5) 


transition: output SEND(elect, 1, 5) in automaton sTreeLeader (1) 
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on drake.csail.mit.edu at 9:29:08:965 
Modified state variables: 


P — Tuple, modified fields: {[send -> Map, modified entries: {[5 -> Sequence, elements adc 
SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> Sequence, elements 


k > 5 
tempNbrs2 — (0) 


transition: output RECEIVE(elect, 7, 11) in automaton sTreeLeader (7) 
on condor.csail.mit.edu at 9:29:09:760 
Modified state variables: 


P — [nbrs: (11 3), receivedElect: (11), sentElect: (3), status: idle, send: Map{[11 -> {}] 


RM — Map{[11 -> [status: idle, toRecv: {}, ready: false]] [3 -> [status: idle, toRecv: 
SM — Map{[11 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [3 -> [status: idle, 
j 3 

k — 11 

rank — 7 

tempNbrs — () 

tempNbrs2 — () 


transition: output SEND(elect, 7, 3) in automaton sTreeLeader (7) 
on condor.csail.mit.edu at 9:29:09:780 
Modified state variables: 


P — Tuple, modified fields: {[send -> Map, modified entries: {[3 -> Sequence, elements adc 
SM — Map, modified entries: {[3 -> Tuple, modified fields: {[toSend -> Sequence, elements 


k > 3 


transition: output RECEIVE(elect, 10, 14) in automaton sTreeLeader (10) 
on loon.csail.mit.edu at 9:29:10:113 
Modified state variables: 


P — [nbrs: (14 6), receivedElect: (14), sentElect: (6), status: idle, send: Map{[14 -> {}] 


RM — Map{[14 -> [status: idle, toRecv: {}, ready: false]] [6 -> [status: idle, toRecv: 
SM — Map{[14 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [6 -> [status: idle, 
jo — 14 

k — 14 

rank — 10 

tempNbrs — () 

tempNbrs2 — (6) 


transition: output SEND(elect, 10, 6) in automaton sTreeLeader (10) 
on loon.csail.mit.edu at 9:29:10:119 
Modified state variables: 


P — Tuple, modified fields: {[send -> Map, modified entries: {[6 -> Sequence, elements adc 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> Sequence, elements 


k > 6 
tempNbrs2 — () 


transition: output RECEIVE(elect, 6, 10) in automaton sTreeLeader (6) 
on drake.csail.mit.edu at 9:29:10:522 
Modified state variables: 


P — [nbrs: (10 2 5), receivedElect: (10), sentElect: (), status: idle, send: Map{[10 -> {] 


RM — Map{[10 -> [status: idle, toRecv: {}, ready: false]] [2 -> [status: idle, toRecv: 
SM — Map{[10 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [2 -> [status: idle, 
j — 10 

k — 10 

rank — 6 

tempNbrs — () 
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tempNbrs2 — () 


transition: output RECEIVE(elect, 
on parrot.csail.mit.edu at 9:29:11:670 


Modified state variables: 
P — [nbrs: 


je 

k — 12 

rank — 8 
tempNbrs — () 
tempNbrs2 — () 


transition: output SEND(elect, 


on parrot.csail.mit.edu at 9:29:11:684 


Modified state variables: 


(12 9), receivedElect: 
RM — Map{[12 -> [status: idle, 
SM — Map{[12 -> [status: idle, 


8, 12) in automaton sTreeLeader (8) 


(12), sentElect: (9), status: idle, send: Map{[12 -> {}] 


toRecv: {}, ready: false]] [9 -> [status: idle, toRecv: {}, 
toSend: {}, sent: {}, handles: {}]] [9 -> [status: idle, t« 


9) in automaton sTreeLeader (8) 


P — Tuple, modified fields: {[send -> Map, modified entries: {[9 -> Sequence, elements adc 


SM — Map, modified entries: 
k > 9 
tempNbrs2 — (12) 


transition: output RECEIVE(elect, 
on parrot.csail.mit.edu at 9:29:12:228 


Modified state variables: 
P — [nbrs: 
RM — Map{[2 -> [status: idle, 
SM — Map{[2 -> [status: idle, 
jhe 

k — 7 

rank — 3 

tempNbrs — () 

tempNbrs2 — () 


transition: output SEND(elect, 


on parrot.csail.mit.edu at 9:29:12:262 
transition: output RECEIVE(elect, 
on tui.csail.mit.edu at 9:29:12:573 


Modified state variables: 
P — [nbrs: 
RM — Map{[5 -> [status: idle, 
SM — Map{[5 -> [status: idle, 
jo 8 

k — 8 

rank — 9 

tempNbrs — () 

tempNbrs2 — () 


Modified state variables: 


(2 7), receivedElect: 


(5 8), receivedElect: 


{[9 -> Tuple, modified fields: {[toSend -> Sequence, elements 


3, 7) in automaton sTreeLeader (3) 


(7), sentElect: (2), status: idle, send: Map{[2 -> felect 


toRecv: {}, ready: false]] [7 -> [status: idle, toRecv: {}, 
toSend: {}, sent: {}, handles: {}]] [7 -> [status: idle, tos 


2) in automaton sTreeLeader (3) 


9, 8) in automaton sTreeLeader (9) 


(8), sentElect: (5), status: idle, send: Map{I[5 -> felect 


toRecv: {}, ready: false]] [8 -> [status: idle, toRecv: {}, 
toSend: {}, sent: {}, handles: {}]] [8 -> [status: idle, tos 


P — Tuple, modified fields: {[send -> Map, modified entries: {[2 -> Sequence, elements adc 


SM — Map, modified entries: 
k > 2 
tempNbrs2 — (7) 


transition: output SEND(elect, 
on tui.csail.mit.edu at 9:29:12:730 


{[2 -> Tuple, modified fields: {[toSend -> Sequence, elements 


5) in automaton sTreeLeader (9) 
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on 


on 


on 


on 


on 


Modified state variables: 

P — Tuple, modified fields: {[send -> Map, modified entries: {[5 -> Sequence, elements adc 
SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> Sequence, elements 
k > 5 

tempNbrs2 — (8) 


Initialization starts (5) on loon.csail.mit.edu at 9:29:12:852 

Modified state variables: 

P — [nbrs: (), receivedElect: (), sentElect: (), status: idle, send: Map{}] 

RM — Map{} 

SM — Map{} 

j — 87 

k — 87 

rank — null 

tempNbrs — () 

tempNbrs2 — () 

Initialization ends 

transition: output RECEIVE(elect, 2, 3) in automaton sTreeLeader (2) 

condor.csail.mit.edu at 9:29:14:748 

Modified state variables: 

P — [nbrs: (3 6), receivedElect: (3), sentElect: (6), status: idle, send: Map{[3 -> {}] Le 
RM — Map{[3 -> [status: idle, toRecv: {}, ready: false]] [6 -> [status: idle, toRecv: {}, 
SM — Map{[3 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [6 -> [status: idle, tos 
{> 3 

k — 3 

rank — 2 

tempNbrs — () 

tempNbrs2 — () 


transition: output SEND(elect, 2, 6) in automaton sTreeLeader (2) 

condor.csail.mit.edu at 9:29:14:759 

Modified state variables: 

P — Tuple, modified fields: {[send -> Map, modified entries: {[6 -> Sequence, elements adc 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> Sequence, elements 
k — 6 


transition: output RECEIVE(elect, 6, 2) in automaton sTreeLeader (6) 
drake.csail.mit.edu at 9:29:15:104 

Modified state variables: 

P — Tuple, modified fields: {[receivedElect -> (10 2)] [sentElect -> (5)] [send -> Map, m 
k— 2 

tempNbrs2 — (10) 


transition: output SEND(elect, 6, 5) in automaton sTreeLeader (6) 
drake.csail.mit.edu at 9:29:15:112 

transition: output RECEIVE(elect, 5, 1) in automaton sTreeLeader (5) 

loon.csail.mit.edu at 9:29:15:145 

Modified state variables: 

P — [nbrs: (1 4 6 9), receivedElect: (1), sentElect: (), status: idle, send: Map{[1 -> {}] 
RM — Map{[1 -> [status: idle, toRecv: {}, ready: false]] [4 -> [status: idle, toRecv: {}, 
SM — Map{[1 -> [status: idle, toSend: {}, sent: {}, handles: {}]] [4 -> [status: idle, tos 
i> 6 

k - il 

rank — 5 

tempNbrs — () 

tempNbrs2 — (6 9) 
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transition: output RECEIVE(elect, 5, 4) in automaton sTreeLeader (5) 
on loon.csail.mit.edu at 9:29:15:492 

Modified state variables: 

P — Tuple, modified fields: {[receivedElect -> (1 4)] } 

k — 4 

tempNbrs2 — () 


Modified state variables: 

P — Tuple, modified fields: {[send -> Map, modified entries: {[5 -> Sequence, elements adc 
SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> Sequence, elements 
k > 5 

tempNbrs2 — (10 2) 


transition: output RECEIVE(elect, 5, 9) in automaton sTreeLeader (5) 
on loon.csail.mit.edu at 9:29:15:836 
Modified state variables: 


P — Tuple, modified fields: {[receivedElect -> (1 4 9)] [sentElect -> (6)] [send -> Map, 1 
k > 9 


transition: output SEND(elect, 5, 6) in automaton sTreeLeader (5) 

on loon.csail.mit.edu at 9:29:16:182 
Modified state variables: 
P — Tuple, modified fields: {[send -> Map, modified entries: {[6 -> Sequence, elements adc 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> Sequence, elements 
k — 6 
tempNbrs2 — (1 4 9) 


transition: output RECEIVE(elect, 5, 6) in automaton sTreeLeader (5) 
on loon.csail.mit.edu at 9:29:16:494 
Modified state variables: 
P — Tuple, modified fields: {[receivedElect -> (1 4 6 9)] } 
SM — Map, modified entries: {[6 -> Tuple, modified fields: {[toSend -> Sequence, elements 


transition: output RECEIVE(elect, 6, 5) in automaton sTreeLeader (6) 
on drake.csail.mit.edu at 9:29:16:554 
Modified state variables: 
P — Tuple, modified fields: {[receivedElect -> (10 2 5)] [status -> elected] } 
SM — Map, modified entries: {[5 -> Tuple, modified fields: {[toSend -> Sequence, elements 


transition: output leader() in automaton sTreeLeader (6) 
on drake.csail.mit.edu at 9:29:16:559 

Modified state variables: 

P — Tuple, modified fields: {[status -> announced] } 


107 


